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1 INTRODUCTION 


This  document  constitutes  the  final  report  on  a project  performed  by 
Southwest  Research  Institute  in  the  area  of  knowledge  acquisition  for 
intelligent  tutoring  systems.  The  focus  of  the  research  has  been  on  the 
development  of  a problem  solving  taxonomy  that  can  support  and  direct  the 
knowledge  engineering  process  during  the  development  of  an  intelligent 
tutoring  system. 


1 . 1 Overview  Of  The  Knowledge  Engineering  Problem 


The  process  of  knowledge  engineering,  the  major  methodology  for 
incorporating  the  knowledge  of  a human  expert  into  a computer  program,  has 
long  been  marked  as  a bottleneck  in  any  knowledge-based  system  development. 
This  is  not  surprising  since  the  process  involves  humans  who  are  expert  in 
a given  domain,  namely  the  domain  experts,  communicating  with  other  humans 
who  are  not  expert  in  the  domain  about  their  expertise  and  problem  solving 
skills.  The  non-domain  experts  are,  however,  expert  in  computer 
programming  and  knowledge  representation  and  they  must  be  able  to  organize, 
structure,  and  convert  the  domain  expertise  as  they  understand  it  into  a 
computer  program.  All  of  this  communication  results  in  a situation  similar 
to  the  "telephone  game,"  where  something  is  lost  or  misinterpreted  with 
each  communication  act.  The  problem,  of  course,  is  magnified  by  the  fact 
that  the  individuals  involved  do  not  tend  to  speak  the  same  "language"  and 
the  concepts  being  communicated  can  be  quite  complex.  A good,  simplified 
description  and  illustration  of  the  knowledge  engineering  process  is  given 
in  Yost  and  Newell  [1989]. 

The  knowledge  acquistion  problem  has  been  acknowledged  since  the  early 
days  of  knowledge-based  system  development.  Many  person-years  of  effort 
would  go  into  the  development  of  a single  knowledge  base.  The  ideal 
solution,  of  course,  is  to  skip  the  knowledge  engineer  and  have  the  domain 
expert  generate  the  knowledge  base  directly.  The  problem  with  this 
approach  is  that  the  domain  expert  usually  has  no  real  knowledge  or 
experience  in  computer  programming  and  knowledge  representation.  Research 
into  the  development  of  tools  to  support  the  knowledge  acquisition  process 
has  been  in  two  directions:  1)  to  support  the  knowledge  engineer  in 
structuring  and  encoding  the  knowledge  acquired  and  2)  to  provide  an 
interface  that  would  allow  the  domain  expert  to  enter  his/her  knowledge 
directly  into  the  computer  program. 

The  first  approach,  targeting  the  knowledge  engineer  as  the  end  user, 
has  generated  all  of  the  knowledge-based  system  development  tools  such  as 
M.l,  S.l,  ART,  KEE,  GoldWorks,  and  CLIPS.  Many  of  these  systems  have 
become  commercial  products  and  are  treated  more  or  less  as  high-level 
programming  languages.  They  have  made  the  knowledge  engineer's  job  easier 
by  providing  a variety  of  knowledge  representation  and  reasoning  paradigms 
as  part  of  the  programming  language,  thus  providing  a structure  into  which 
the  knowledge  acquired  can  be  entered. 
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The  second  approach,  targeting  the  domain  expert  as  the  end  user,  has 
generated  knowledge  acquisition  systems  such  as  RuleMaster,  TIMM, 
TEIRESIAS,  MOLE,  ROGET,  and  ETS.  Most  of  these  systems  have  not  become 
commercial  products,  indicating  clearly  that  we  do  not  yet  understand  the 
knowledge  acquistion  problem  well  enough  to  develop  truly  viable  systems. 
Some  of  these  systems  generalize  from  examples  while  others  query  the 
expert  directly,  either  about  specific  knowledge  such  as  class  hierarchies 
and  relations  or  about  more  general  problem  solving  techniques.  Each 
approach  has  its  advantages  and  drawbacks.  The  major  trade-offs  involve 
less  general  elicitation  techniques  that  work  on  only  a single  domain  but 
that  produce  systems  with  a high  level  of  expertise  vs.  more  general 
elicitation  techniques  that  work  on  a range  of  domains  but  produce  systems 
that  do  not  have  the  detailed  knowledge  needed  to  solve  problems  at  the 
expert  level.  This  is  a classic  trade-off  problem  and  has  appeared 
continuously  in  one  form  or  another  since  the  early  days  of  AI  research  and 
development . 


1.2  Traditional  Knowledge  Acquisition  Techniques 


The  literature  on  knowledge  acquisition  discusses  four  basic  verbal 
knowledge  elicitation  techniques: 

1.  interviews 

2.  observational  studies 

3.  retrospective  methods 

4.  verbal  protocols 

Knowledge  can  also  be  pulled  from  written  sources,  such  as  textbooks  and 
manuals.  These  methods  elicit  different  and  sometimes  complementary  types 
of  knowledge.  Most  knowledge  engineers  employ  a variety  of  techniques  to 
build  a knowledge  base. 

Knowledge  engineers  often  use  interview  techniques  in  early  sessions 
with  the  domain  experts.  Interviews  can  be  descriptive,  problem-oriented, 
or  structured.  A descriptive  interview  is  usually  the  first  interaction 
technique  applied  in  knowledge  elicitation.  The  expert  presents  a 
structured  description  of  the  domain  to  the  knowledge  engineer.  The 
interview  is  similar  to  a lecture  and  familiarizes  the  knowledge  engineer 
with  the  basic  domain  concepts.  The  expert  may  recommend  documents  for  the 
knowledge  engineer  to  study  on  his/her  own.  Knowledge  engineers  structure 
problem-oriented  interviews  to  obtain  specific  knowledge  from  the  expert. 
The  knowledge  engineer  presents  the  expert  with  a problem  and  asks 
predetermined  questions  at  each  step  in  the  problem  solution.  Finally, 
structured  interviews  center  around  inputting  domain  knowledge  into  a 
knowledge  representation  scheme,  such  as  a hierarchy  of  frames  (Peterson, 
et.  al,  1988). 

Interview  techniques  are  useful  for  eliciting  basic  domain  knowledge 
or  filling  in  gaps  from  knowledge  gathered  with  other  techniques,  but  they 


do  not  capture  the  expert's  problem-solving  strategies  and  detailed  or 
inaccessible  domain  knowledge. 

In  observational  studies,  the  knowledge  engineer  observes  the  expert 
engaged  in  daily  problem-solving  activities.  The  expert  follows  his/her 
normal  daily  schedule  and  the  knowledge  engineer  simply  videotapes  the 
expert  during  problem-solving  and  in  face-to-face  or  phone  interactions 
with  colleagues.  This  technique  provides  information  about  the  expert's 
role  in  the  problem-solving  process,  but  it  seldom  yields  much  detailed 
domain  knowledge. 

Retrospective  techniques  require  the  expert  to  recall  an  incident  in 
which  he/she  solved  a domain  problem  in  as  much  detail  as  possible.  This 
methodology  provides  a detailed  analysis  of  the  expert's  problem  solving 
process,  but  is  impaired  by  the  problems  of  human  recall.  Humans  tend  to 
reconstruct  events  in  recall  and  often  report  events  differently  than  they 
occurred  (Bartlett,  1932;  Bransford  and  Franks,  1971;  Veisberg,  1980).  The 
advantage  of  retrospective  methods  is  that  they  yield  detailed  descriptions 
of  the  expert's  problem-solving  strategies  and  the  alternative  choices 
considered. 

Despite  advances  in  automated  knowledge  acquistion  techniques, 
protocol  methodologies  (Ericsson  and  Simon,  1984)  are  still  the  most 
frequently  used  method  of  knowledge  acquisition  in  practical  applications. 
And  although  these  methods  are  demanding  and  time  consuming  for  knowledge 
engineers  and  experts,  they  demand  less  of  experts  than  most  automated 
techniques.  An  expert  simply  solves  a domain  problem  while  speaking  aloud 
and  the  knowledge  engineer  records  the  session.  The  expert  is  not 
obligated  to  spend  many  hours  interacting  with  an  unfamiliar  computer 
system. 

Collecting  a protocol  allows  the  knowledge  engineer  to  observe  the 
ongoing  problem-solving  process  and  capture  detailed  information  about  the 
domain  knowledge  used  by  the  expert.  This  method  also  permits  the 
knowledge  engineer  to  infer  knowledge  that  is  used  by  the  expert  but  not 
directly  verbalized. 

Protocol  techniques  have  two  primary  disadvantages.  Verbalizing 
knowledge  during  problem  solving  may  interfere  with  the  task  or  it  may 
alter  the  expert's  usual  approach.  And,  because  at  no  time  during  protocol 
collection  does  the  knowledge  engineer  disrupt  the  process,  knowledge  about 
problem-solving  strategies  and  alternatives,  as  seen  in  retrospective 
techniques,  is  not  available. 

In  practice,  verbal  methods  require  much  more  labor  than  a surface 
description  implies.  The  knowledge  engineer  generally  employs  a wide 
variety  of  verbal  methods  and  meets  with  the  domain  expert  many  times 
throughout  the  course  of  system  development,  refining  and  adding  to  the 
emerging  knowledge  base.  Because  the  knowledge  engineer  is  generally  not  a 
domain  expert,  he/she  relies  on  the  domain  expert  to  supply  all  the 
critical  domain  knowledge.  This  can  lead  to  a variety  of  problems  as 
listed  below. 
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1.  The  expert  often  has  no  sense  of  whether  a piece  of  information  is 
critical  and  may  neglect  to  mention  some  important  aspect  of  problem 
solving  until  late  in  the  development  process.  This  could  cause  the 
entire  structure  of  the  knowledge  base  to  change. 

2.  One  characteristic  of  expertise  is  that  problem  solving  in  the  domain 
is  such  a well-learned  task  that  the  expert  is  no  longer  consciously 
aware  of  all  the  steps  to  a problem  solution.  Thus,  the  expert  may  be 
unable  to  report  certain  critical  activities. 

3.  In  the  same  vein,  introspective  reports  of  cognitive  activity  are 
notoriously  inaccurate,  suffering  from  memory  biases,  reporting  biases, 
and  inability  to  verbalize  all  aspects  of  the  process. 

4.  Finally,  experts  bring  heuristics  to  the  problem  solving  process  that 
are  based  on  seemingly  simple  yet  powerful  analytical  models  derived 
from  experience  with  the  domain.  The  expert  may  have  little 
justification  for  these  heuristics  other  than  "they  work."  The 
knowledge  engineer  is  faced  with  the  task  of  encoding  these  heuristics 
and  infering  information  essential  to  building  a working  system. 

All  of  these  are  issues  with  the  knowledge  acquisition  process,  whether  the 

process  is  performed  by  humans  or  by  an  automated  tool. 


1.3  State-of-the-Art  In  Knowledge  Acquisition 


Host  of  the  systems  that  might  be  called  automatic  knowledge 
acquisition  tools  have  been  directed  at  providing  a means  for  the  computer 
to  elicit  the  needed  knowledge  directly  from  the  domain  expert  and  to  put 
it  into  the  knowledge  base  of  an  expert  system.  Much  of  the  literature  on 
such  systems  appears  in  what  is  considered  the  machine  learning  area  of  AI. 
How  such  systems  acquire  the  knowledge  from  the  experts  can  generally  be 
classified  into  two  types:  1)  those  that  create  generalized  rules  from 
specific  examples,  called  induction-based  learning  systems  and  2)  those 
that  fill-in  knowledge  structures,  such  as  templates,  based  on  querying  the 
user  in  some  structured  fashion,  usually  referred  to  as  interview-based 
systems . 

Induction-based  systems,  such  as  RuleMaster,  TIMM,  and  Expert  Ease, 
look  similar  to  a spreadsheet  in  terms  of  their  user  interface.  They  work 
by  having  the  user  enter  example  situations  by  identifying  salient  features 
of  the  problem,  providing  specific  values  for  those  features  for  each 
example,  and  giving  the  correct  answer  to  the  problem  for  each  example. 
Through  logically-based  induction  techniques,  general  rules  are  then 
developed  by  the  system.  The  result  is  a system  that  can  function  quite 
well  on  any  problem  that  is  the  same  as  the  examples  the  system  was  trained 
on,  but  that  can  have  fairly  unpredictable  behavior  on  those  problems  that 
were  not  the  same  as  the  ones  trained  on.  This  kind  of  approach  works  in 
domains  where  problems  can  be  easily  posed  as  sets  of  salient  features  and 
values,  and  possible  solutions  can  be  enumerated,  such  as  structured 
selection  problems.  However,  input  becomes  rather  unwieldy  if  many 
features,  values,  and  solutions  exist  in  the  problem  domain,  because  the 


number  of  possible  example  cases  grows  exponentially  with  each  additional 
feature  and  potential  value.  Human  experts  often  do  think  in  terms  of 
example  cases,  but  usually  only  in  terms  of  their  similarities  and 
differences  with  respect  to  the  current  problem  under  consideration.  In 
most  complex  domains,  human  experts  have  already  made  a lot  of 
generalizations  and  those  could  be  acquired  more  efficiently  through  direct 
query  methods. 

Interview-based  systems,  such  as  THEIRESIAS  (Davis,  1982),  HOLE 
(Eshelman  and  McDermott,  1986),  MORE  (Kahn  et  al.,  1985),  KNACK  (Klinker  et 
al.,  1987),  SALT  (Marcus  et  al.,  1985),  ROGET  (Bennet,  1983),  ETS  (Boose, 
1985),  and  ASKE  (Patel,  1989),  attempt  to  determine  the  knowledge  content 
of  a domain  by  querying  the  domain  expert  based  on  some  structured 
methodology.  For  example,  MOLE  elicits  symptom-hypothesis  linkages  and  the 
type  and  strength  of  the  linkages  from  the  domain  expert  in  an  attempt  to 
build  a diagnostic  system.  ETS  uses  a methodology  stemming  from  personal 
construct  theory  to  query  the  domain  expert  concerning  the  distinctions 
that  exist  between  possible  conclusions  that  an  expert  can  make  and  uses 
distance-based  cluster  analysis  to  build  concept  hierarchies, 
discrimination  networks,  and  decision  trees.  Most  of  these  knowledge 
acquisition  systems  are  oriented  towards  the  building  of  knowledge-based 
systems  that  perform  structured  selection  problem  solving,  such  as 
diagnostics.  Two  systems,  KNACK  and  SALT,  do  address  slightly  different 
kinds  of  problems,  namely  design  evaluation  and  solution  construction, 
respectively.  However,  their  basic  knowledge  acquisition  strategies  are 
still  to  construct  taxonomies  of  concepts  and  concept  properties. 

Thus,  the  work  to  date  in  the  area  of  automated  knowledge  acquisition 
tools  has  been  heavily  oriented  towards  the  structured  selection  problem 
solving  paradigm.  This  is  not  surprising  since  a lot  of  the  work  in 
knowledge-based  systems  has  also  been  oriented  towards  this  kind  of  problem 
solving  task.  Class  hierarchies,  and  the  relationships  between  them,  have 
been  a dominant  goal  of  these  knowledge  acquisition  systems,  as  well  as 
generalized  if-then  rules.  This  kind  of  knowledge  fits  very  well  into  the 
current  knowledge  representation  schemes  available  in  the  knowledge-based 
system  development  tools  on  the  market  today,  such  as  frames,  semantic 
networks,  and  production  rules. 


1.4  Foundation  For  The  Next  Generation  Knovledge  Acquisition  Tools 


Completely  automated  knowledge  acquisition  tools  that  allow  a domain 
expert  to  generate  a sophisticated,  complex  knowledge-based  system  with  no 
support  from  an  individual  knowledgeable  in  computer  programming  and 
artificial  intelligence  is  a lofty  goal.  Current  understanding  of  human 
learning  and  cognition  is  not  at  a level  to  allow  the  development  of 
completely  general,  domain  independent  automated  knowledge  acquisition 
tools  capable  of  developing  arbitrarily  complex  knowledge-based  systems  in 
any  domain.  In  addition,  most  domain  experts  would  not  have  the  time  or 
inclination  to  submit  themselves  to  extensive,  tedious  question-answer 
sessions  with  a computer  program.  Another  approach  to  the  knowledge 
acquisition  problem,  that  leaves  the  human  knowledge  engineer  in  the  loop, 
might  be  a better  short-term  solution. 
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It  has  been  readily  acknowledged  among  the  practitioners  that  the 
knowledge  engineering  process,  as  performed  currently,  is  more  an  art  than 
a science.  Of  course,  there  are  techniques  with  names  attached  to  them 
such  as  structured  and  unstructured  interviews,  example  cases,  etc.,  and 
there  are  recommended  practices  such  as  taping  the  interviews,  generating 
transcripts,  and  then  editing  and  modifying  the  resulting  knowledge 
acquired  based  on  domain  expert  feedback.  However,  the  quality  of  the 
resulting  system  is  still  highly  dependent  on  the  personalities  and  skills 
of  the  individuals  involved.  Successful  knowledge  engineers,  aside  from 
having  a sound  computer  science  background  and  an  understanding  of  AI 
techniques,  must  also  be  intelligent,  extremely  inquisitive,  good  problem 
solvers,  well-organized,  and  have  excellent  communication  and  interpersonal 
skills.  These  knowledge  engineers  must  be  capable  of  putting  people  at 
ease  during  a knowledge  engineering  session  while  at  the  same  time  asking 
them  to  think  and  introspect  about  how  experts  solve  problems  in  their 
particular  area  of  expertise.  The  key  to  a successful  knowledge 
engineering  process  (i.e.,  a useful,  working  knowledge-based  system)  is  the 
ability  to  extract  the  general  problem  solving  methodologies  utilized  and 
the  specific  domain  knowledge  needed  to  find  a solution  to  a given  problem. 

Thus,  the  goal  of  the  project  described  in  this  report  for  the 
knowledge  acquisition/knowledge  engineering  problem  was  to  develop  a 
methodology  that  could  be  used  first  to  typify  the  expert  problem  solving 
strategies  and  the  types  of  specific  domain  knowledge  needed  that  would 
allow  the  knowledge  engineer  to  categorize  the  task.  An  initial 
categorization  could  then  provide  guidelines  for  understanding  the  key 
characteristics  of  the  given  kind  of  task.  These  techniques  could 
eventually  be  automated,  but  initial  experience  with,  and  evaluation  of, 
the  techniques  should  be  gained  through  the  use  of  good  human  knowledge 
engineers . 

The  approach  to  generating  such  methodologies  and  classes  was  to  a 
study  of  a broad  spectrum  of  problem  solving  tasks,  analyze  the  problem 
solving  techniques  used,  and  a pair  of  the  knowledge  engineering  techniques 
to  the  acquisition  and  codification  of  specific  problem  solving  strategies 
into  a knowledge-based  system.  The  result  of  this  process  is  a proposed 
taxonomy  of  problem  solving  tasks  characterized  by  problem  solving 
strategies  and  types  of  domain  knowledge,  as  well  as  some  recommendations 
concerning  knowledge  acquisition  methodologies  relevant  to  the  particular 
task.  Some  work  in  this  area  appears  in  the  literature  (see  Section  2.2). 


1.5  Project  Rationale  And  Approach 


The  research  described  in  this  report  involves  identifying  and 
analyzing  a pre-defined  number  of  problem  solving  tasks.  These  tasks  were 
representative  of  the  variety  of  tasks  that  are  of  interest  to  the  Air 
Force  and  NASA  with  respect  to  issues  of  training.  The  tasks  included 
diagnostics,  design,  data  interpretation,  procedure  execution,  planning, 
categorization,  monitoring,  etc.  The  complete  list  could  be  extensive,  but 
this  effort  attempted  to  identify  representative  tasks,  approximately  36, 
and  was  by  no  means  exhaustive. 
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The  intended  benefits  and  goals  of  the  research  include  the  following! 

o Gain  insight  into  the  types  of  problem  solving  tasks  that  are  of 

interest  to  the  Air  Force  and  NASA  in  training  and  what  characterizes 

these  types  of  problems. 

o Provide  a viable  approach  to  the  knowledge  engineering  process  that 

gives  some  direction  to  the  knowledge  engineer  concerning  how  to 
interview  a domain  expert  in  a given  problem  area. 

o Provide  a framework  in  which  a comparison  of  approaches  and  techniques 

for  knowledge  engineering  can  be  made. 

Based  on  the  study  that  has  been  performed  and  is  reported  in  this 
document,  these  goals  have  been  partially  met,  though  considerably  more 
research  could  be  performed  to  develop  the  ideas  further  and  test  the 
hypotheses  that  were  generated  by  this  study. 

The  work  performed  involved  identifying  a set  of  representative  tasks, 
interviewing  experts  in  the  selected  tasks,  classifying  these  tasks  into  a 
taxonomy  based  on  their  characteristics,  and  generating  recommendations  for 
performing  knowledge  acquisition  based  on  the  task  classification.  In  some 
sense,  the  work  to  develop  a taxonomy  needed  to  be  done  before  we  could 
make  sense  of  the  results  gained  from  the  knowledge  acquisition  interviews. 
But,  at  the  same  time,  the  knowledge  acquisition  interviews  supplied  the 
information  on  tasks  that  could  support  the  generation  of  a taxonomy. 
Thus,  because  this  was  an  initial  research  project,  a bootstrapping 
approach  had  to  be  taken  to  get  any  kind  of  results.  Therefore,  interviews 
were  performed  in  each  task  area.  This  allowed  us  to  gain  some  insight 
into  the  problem  solving  tasks,  generate  some  hypotheses  and  classification 
schemes,  and  then  informally  test  these  hypotheses  and  classifications 
through  a second  interview.  Further  details  on  these  steps  are  given  in 
the  rest  of  this  report. 

The  approach  taken  in  this  research  has  been  a very  pragmatic  one.  An 
outline  of  the  overall  process  we  followed  is  provided  in  Figure  1.  The 
work  has  been  based  on  years  of  experience  in  performing  knowledge 
acquisition  for  the  design  and  implementation  of  numerous  knowledge-based 
and  intelligent  tutoring  systems  in  a wide  variety  of  problem  solving 
domains.  One  of  the  underlying  goals  has  been  to  minimize  the  amount  of 
time  required  with  the  expert.  Experience  has  shown  that  an  expert's  time 
is  usually  very  limited  and  it  helps  to  make  the  most  of  the  time  that  you 
do  have.  Another  underlying  goal  of  the  research  was  to  develop  an 
approach  to  knowledge  acquisition  that  would  help  a knowledge  engineer 
characterize  the  problem  solving  task  in  some  useful  way  and  to  scope  the 
effort.  Thus,  much  of  the  questioning  is  oriented  towards  acquiring 
knowledge  of  the  inputs  to  the  task,  the  results  or  output  from  the  task, 
reasoning  mechanisms  that  operate  on  the  input  and  generate  the  output,  the 
environment  in  which  the  problem  solving  takes  place,  and  the  attributes  of 
the  experts  who  perform  the  task.  The  effort  was  less  concerned  with  total 
accuracy  of  the  knowledge  obtained  than  with  discovering  the  problem 
solving  approach(es)  used.  Accuracy  can  come  later  when  the  software  is 
under  development  and  the  expert  can  observe  the  system's  behavior  and 
suggest  concrete  corrections. 


STEP  t 
STEP  2 
STEP  3 
STEP  4 
STEP  5 
STEP  6 
STEP  7 
STEP  8 
STEP  9: 


: Obtain  set  of  task  characteristics 
: Choose  set  of  tasks 

: Design  initial  knowledge  acquisition  interview 
: Conduct  interviews  for  each  task 

: Evaluate  initial  interview  and  construct  preliminary  classification  of  tasks 
: Design  follow-on  knowledge  acquisition  procedure  based  on  task  characteristics 
: Conduct  second  knowledge  acquisition  session 
: Evaluate  second  interview 
Construct  proposed  knowledge  acquisition  taxonomy 


Figure  1.  Knowledge  acquisition  study  methodology 
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This  study  generated  a number  of  hypotheses  about  how  to  conduct 
knowledge  acquistion  for  a variety  of  tasks  and  therefore  provides  some 
direction  in  how  knowledge  acquisition  should  proceed.  However,  the 
hypotheses  need  to  be  verified  through  more  formal  experimentation  than  was 
possible  in  this  limited  effort.  Results  of  the  research  and 
recommendations  concerning  what  should  be  done  next  are  given  in  Section  7. 


BW  laffaftT  w;<AU.  t 


2 PROBLEM  SOLVING  TAXONOMIES  AND  TASK  CHARACTERISTICS 


Practical  experience  in  knowledge-based  system  development  indicates 
that  much  of  the  system  design  depends  on  the  characteristics  of  the  task, 
that  the  knowledge-based  system  needs  to  perform  or  teach.  Researchers  in 
artificial  intelligence  (AI)  are  beginning  to  evaluate  the  relationship 
between  knowledge  representation,  system  architecture,  and  task  type.  This 
work  is  related  to  the  growing  awareness  that  the  method  of  knowledge 
acquisition  should  depend  on  the  characteristics  of  the  task  and  to  the 
continued  focus  in  the  psychological  and  educational  literature  on  the 
relationship  between  task  characteristics  and  instructional  strategy  in 
curriculum  development  and  on  the  relationship  between  task  characteristics 
and  human  factors  engineering. 

The  growing  focus  on  importance  of  task.  characteristics  for 
knowledge-based  system  design  signals  the  maturation  of  the  field  from  one 
searching  for  the  single  best  representational  format  and  inferencing 
strategy  to  one  that  recognizes  the  diversity  of  problems  and  the  need  to 
address  each  according  to  its  characteristics.  Not  surprisingly,  human 
cognition  appears  to  utilize  multiple  representational  formats  and 
reasoning  techniques.  Different  representations  store  different  types  of 
information  (e.g.,  scripts  store  simple,  well-structured  sequences  of 
events)  and  dictate  the  appropriate  methods  for  reasoning  about  that 
information.  The  flexibility  to  employ  the  best  representation  and 
reasoning  strategy  for  a given  problem  undoubtedly  contributes  to  human 
intelligence. 

Research  in  knowledge  acquisition  has  implicitly  recognized  the  need 
to  tailor  the  knowledge  acquisition  process  to  the  task  characteristics. 
For  example,  structured  interviews  center  around  filling  knowledge  into  the 
chosen  representational  format.  As  such,  the  format  determines  the 
knowledge  that  is  needed  and  the  reasoning  strategies  that  will  be 
effective.  Knowledge  engineers  who  use  this  strategy  correctly  look  to 
their  experience  to  select  the  representation  format  based  on  the  problem 
type.  However,  much  too  often  the  cart  comes  before  the  horse,  i.e.,  the 
representation  is  selected  and  the  problem  is  forced  to  fit  to  the 
representation. 

In  addition,  most  automated  knowledge  acquisition  tools  are  specific 
to  a problem  type,  such  as  diagnosis  or  design  (although  some  such  as 
AQUINAS  claim  to  be  more  general).  For  example,  tools  such  as  MOLE 
(Eshelman,  Ehret,  McDermott  and  Tan,  1987),  TEIRESIAS  (Davis,  1985),  ROGET 
(Bennett,  1983),  and  TKAW  (Kahn,  Breaux,  Joseph,  and  Deklerk,  1987)  acquire 
knowledge  about  diagnostic  application  tasks.  While  systems  such  as  SALT 
(Marcus,  1987;  Marcus,  McDermott,  and  Wang,  1985)  and  OPAL  (Musen,  Fagan, 
Combs,  and  Shortliffe,  1987)  collect  knowledge  about  design  and  planning 
tasks,  respectively. 

Recently,  the  relationship  of  task  characteristics  to  the  knowledge 
acquisition  process  has  become  more  explicit.  McDermott  (1988)  has 
proposed  a preliminary  taxonomy  of  problem-solving  methods  and  described 
how  those  methods  can  be  mapped  onto  tasks.  By lander  and  Chandrasekaran 
(1987)  have  suggested  that  generic  tasks  can  serve  to  focus  knowledge 
acquisition.  And  Gaines  (1987)  has  suggested  a hierarchy  of  knowledge 
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transfer  techniques  based  on  areas  of  application.  . 

The  study  presented  in  this  paper  addresses  the  characteristics  of 
tasks  in  an  attempt  to  formulate  a method  for  formally  describing  tasks  in 
support  of  knowledge  acquisition.  The  first  step  in  the  study  was  to 
evaluate  methods  of  characterizing  tasks.  In  order  to  do  so,  we  reviewed 
the  literature  on  factors  affecting  human  performance  and  on  the 
relationship  between  knowledge  acquisition  and  task  type.  The  following 
sections  discuss  the  details  of  the  review. 


2.1  Factors  Affecting  Human  Performance 


A number  of  researchers  over  the  years  have  developed  taxonomies  of 
factors  that  affect  human  performance.  Not  all  of  the  taxonomies  produced 
included  a comprehensive  listing  of  all  relevant  factors.  Some  focused  on 
a sub-set  of  the  factors  affecting  human  performance.  However  taken 
together,  these  taxonomies  allow  researchers  to  characterize  the  cognitive 
operations  that  transpire  when  a person  performs  a task.  They  also  allow  a 
complete  characterization  of  the  environment  in  which  the  task  is  performed 
and  the  characteristics  of  the  subject  who  performs  the  task.  Gawron, 
Drury,  Gzaja,  and  Wilkins  (1989)  combined  the  existing  taxonomies  to 
provide  a comprehensive  one.  They  first  reviewed  taxonomies  of  information 
processing  factors  that  affect  human  performance.  The  goal  of  their  review 
was  to  form  a composite  taxonomy  based  on  the  categories  included  in  human 
performance  taxonomies  over  the  years. 

One  early  taxonomy  was  Miller's  (1962)  taxonomy  of  the  behaviors 
associated  with  the  performance  of  a task,  relying  on  behavior  analysis  for 
task  modeling.  As  a behaviorist,  he  did  not  discuss  cognitive  operations, 
but  he  did  include  some  task  characteristics  that  he  considered  covert 
behavior,  e.g.,  short-term  retention.  These  behaviors  could  be  inferred 
but  not  directly  observed.  His  taxonomy  made  no  allowance  for  the 
influence  of  environment  on  task  performance.  A similar,  more  detailed 
taxonomy  was  developed  by  Berliner,  Angell,  and  Shearer  (1964). 

A taxonomy  by  Fitt  (1965)  recognized  that  the  human  is  embedded  in  the 
environment  and  that  stimulus  characteristics  affect  human  performance. 
His  taxonomy  included  nervous  system  activity,  such  as  receptor,  central 
nervous  system  (CNS),  and  effector  activity,  as  well  as  the  relationship  of 
the  task  to  the  environment,  control,  machine,  and  display  characteristics. 
Thus,  this  taxonomy  recognizes  that  task  performance  involves  perceptual, 
cognitive,  and  motor  activity  mediated  by  environmental  variables.  Fitt's 
taxonomy  was  not  very  detailed  and  other  authors  have  elaborated  on  the 
categories  in  his  taxonomy  (Blanchard,  1973;  Willis,  1961;  Chambers,  1973; 
Hindmarch,  1980). 

Willis  (1961)  provided  a detailed  breakdown  of  nervous  system  input 
and  output.  Receptor  activity  involves  discriminating  non-verbal  cues  and 
discriminating  verbal  cues.  CNS  activity  includes  recall  of  information  of 
facts,  principles,  and  procedures  and  symbolic  data  operations.  Finally, 
effector  activity  is  broken  into  skilled  motor  acts  and  overt 
verbalization.  This  was  one  of  the  earliest  taxonomies  to  explicitly 


include  cognitive  operations  and  be  based  on  the  information  processing 
perspective  of  human  cognition. 

Gagne  (1974)  focused  on  central  nervous  system  processes,  including 
intellectual  skills,  cognitive  strategies,  verbal  information,  and 
attitude.  Intellectual  skills  involve  generating  a solution  to  a novel 
problem,  applying  a rule,  classifying  an  object,  identifying  a class,  and 
discriminating  between  objects.  Willis'  and  Gagne's  taxonomies  highlight 
the  importance  of  the  cognitive  representations  that  support  human 
performance. 

Other  researchers  have  developed  special  purpose  taxonomies.  For 
example,  Shingledecker , Crabtree  and  Acton  (1982)  developed  a taxonomy  for 
indentifying  workload  in  a fighter  aircraft  that  includes  very  specific 
tasks,  such  as  flight  decision  assessment.  Meyer,  Laveson,  Pape,  and 
Edwards  (1978)  developed  a different  taxonomy  for  the  same  environment. 
Carter  (1986)  developed  a taxonomy  for  computer  functions,  including 
categories  such  as  information  functions,  task  execution  functions, 
peripheral  functions,  and  data  functions. 

Gawron,  et.  al.  combined  these  taxonomies  and  added  independent 
variables  manipulated  in  psychological  studies,  such  as  stimulus 
attributes,  and  added  individual  difference  variables,  such  as  gender  and 
age,  to  create  the  Human's  taxonomy.  This  taxonomy  is  intended  to  be  a 
comprehensive  taxonomy  of  factors  in  human  performance.  The  taxonomy 
contains  three  top  level  categories  (environment,  subject,  and  task 
characteristics)  and  has  a depth  of  up  to  eight  levels  (see  Appendix  P). 

Kyllonen  and  Shute  (1989)  approached  their  review  of  taxonomies  from  a 
different  perspective.  They  were  more  interested  in  taxonomies  of  learning 
skills  rather  than  factors  that  affect  human  performance  in  all  types  of 
tasks.  Their  review  argued  that  taxonomies  of  learning  skills  fall  into 
three  categories: 

1.  rational  taxonomies  based  on  a rational  analysis  of  the  domain, 

2.  correlational  taxonomies  based  on  an  individual-differences  analysis, 

3.  model-based  taxonomies  based  on  formal  models  of  learning  processes. 

Rational  taxonomies  are  frequently  based  on  a condi tions-of-learning 
analysis  of  the  domain,  defining  tasks  in  relation  to  characteristics  that 
help  or  hinder  learning.  Melton  (1964)  proposed  a taxonomy  based  on 
learning  tasks  popular  at  the  time:  conditioning,  rote  learning, 
probability  learning,  skill  learning,  concept  learning,  and  problem  solving 
(see  also  Estes,  1982).  Jensen  (1967)  extended  this  taxonomy  to  account 
for  the  method  of  the  study  and  the  type  of  stimuli.  His  taxonomy  included 
a learning  type  facet,  corresponding  to  Melton's  categories,  a procedures 
facet  for  methodological  variables  such  as  the  pacing  of  the  task  and  type 
of  practice,  and  a content  facts  to  indicate  whether  the  stimuli  were 
verbal,  numerical,  or  spacial.  Gagne's  (1974)  learning  outcome  taxonomy, 
discussed  earlier,  is  another  example  of  a rational  taxonomy. 

Kyllonen  and  Shute  argue  that  rational  taxonomies  are  subject  to 
imprecision  because  they  are  not  based  on  a strong  theoretical  model  of 
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learning  and  have  no  foundation  in  empirical  relationships.  These 
taxonomies  do  not  include  actual  psychological  process  dimensions  and, 
according  to  Kyllonen  and  Shute,  process  dimensions  are  the  most  important 
commonality  between  tasks. 

Correlational  taxonomies  stem  from  empirical  results  examining  the 
correlation  between  tasks.  Highly  correlated  tasks  are  assumed  to  require 
similar  learning  skills.  Underwood,  Boruch,  and  Malmi  (1978)  examined  the 
intercorrelation  between  paired  associate,  free  recall,  serial  recall, 
memory  span,  and  frequency  judgment  memory  tests.  Most  of  the  variance  was 
due  to  individual  differences  in  associative  learning  related  to  type  of 
task  (free  recall  versus  paired  associates  and  serial  learning). 

Other  correlational  studies  have  examined  a wider  variety  of  learning 
tasks  (Stake,  1961;  Allison,  1960).  Kyllonen  and  Shute  criticize  the 
correlational  approach  because  there  is  no  a priori  reason  for  selecting 
tasks  to  study,  thus  approaching  theory  building  from  the  bottom  up  (i.e., 
using  experimental  results  to  drive  the  theory  building  process  which 
deviates  from  the  normal  scientific  process  of  developing  theories  that  can 
be  disproven  experimentally).  While  correlational  studies  may  be  useful  to 
explore  the  relationships  between  tasks,  they  cannot  cover  a complete  set 
of  tasks  because  there  is  no  theoretical  basis  for  including  tasks  in  the 
study.  Finally,  any  study  attempting  to  sample  a large  enough  diversity  of 
tasks  to  begin  to  generate  a comprehensive  taxonomy  would  be  prohibitively 
expensive. 

Model-based  taxonomies  are  not  new.  Teichner  (1970)  developed  a 
taxonomy  of  tasks  based  on  information  theory.  What  is  new  is  the  use  of 
cognitive  models  stemming  from  the  information-processing  perspective  to 
drive  the  development  of  taxonomies  of  learning  skills.  One  model-based 
taxonomy  stems  from  Anderson's  (1983)  ACT  (Adaptive  Control  of  Thought) 
theory.  This  theory  proposes  two  fundamental  forms  of  knowledges 
procedural  knowledge  and  declarative  knowledge.  Procedural  knowledge  is 
knowledge  of  how  to  perform  an  operation,  and  declarative  knowledge  is 
factual  knowledge.  ACT  specifies  that  declarative  is  knowledge  is  learned 
by  the  probabilistic  transfer  of  information  to  long-term  memory  and  that 
procedural  knowledge  is  learned  by  knowledge  compilation. 

Another  area  in  which  model-based  taxonomies  have  arisen  is  in  machine 
learning.  Researchers  in  this  area  have  developed  taxonomies  of  learning 
strategies.  For  example,  Carbonell,  et.  al.  (1983)  proposed  a taxonomy 
of  learning  strategies  consisting  of  rote  learning,  learning  from 
instruction,  learning  by  drill  and  practice,  and  inductive  learning. 
Michalski  (1986)  extended  this  to  include  learning  by  deduction. 

From  the  ACT  and  learning  strategies  model-based  taxonomies,  Kyllonen 
and  Shute  proposed  a taxonomy  of  learning  consisting  of  four  dimensions: 


1.  Instructional  Environment  - including  learning  by  rote,  discovery, 
practice,  analogy,  examples,  and  didactic  learning 

2.  Knowledge  Type  - including  propositions,  schema,  rules,  general  rules, 
skills,  general  skills,  automatic  skills,  and  mental  models. 
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3.  Domain  - the  degree  of  quantitativeness  and  the  importance  of  quality 
versus  speed  in  decision  making 

4.  Learning  systems  - including  a variety  of  dimensions  such  as 

holistic/serial  processing,  theory/data  driven,  spatial/verbal 
representation,  etc. 

In  addition  to  capturing  the  type  of  knowledge  and  instruction  involved  in 
a learning  task,  this  taxonomy  captures  characteristics  of  the  environment, 
the  subject  material,  and  the  learner.  The  authors  discuss  how  their 
taxonomy  could  be  applied  to  instructional  system  development. 

In  the  study  described  in  this  report,  we  selected  characteristics 
from  the  comprehensive  taxonomy  generated  by  Gawron,  et.  al.  This 

taxonomy  includes  most  of  the  knowledge  type,  domain,  and  learning  systems 
dimensions  discussed  by  Kyllonen  and  Shute. 


2.2  Knowledge  Acquisition  And  Task  Type 

Knowledge  acquisition  is  another  area  in  which  relativism  has  come  of 
age.  Recently  researchers  have  become  more  and  more  interested  in  basing 
their  selection  of  a knowledge  acquisition  technique  on  the  characteristics 
of  the  task.  The  ultimate  goal  is  to  find  a way  to  attenuate  the 
well-known  knowledge  acquisition  bottleneck.  Gaines  (1987)  reviewed  the 
major  problems  associated  with  expertise  elicitation: 

1.  Expertise  may  not  be  available  to  awareness. 

2.  Expertise  may  not  be  expressible  in  language. 

3.  Expertise  may  not  be  understandable  when  expressed  in  language. 

4.  Expertise  may  not  be  applicable  even  when  expressed  in  language. 

5.  Expertise  expressed  may  be  irrelevant. 

6.  Expertise  expressed  may  be  incomplete. 

7.  Expertise  expressed  may  be  incorrect. 

8.  Expertise  may  depend  on  situational  variable. 

Presumably,  tailoring  the  knowledge  acquisition  process  to  the  task  will 
reduce  the  problems  associated  with  eliciting  expertise. 

Clancey  (1986)  proposed  a taxonomy  of  tasks  and  suggested  that  two 
fundamental  problem  solving  methods  could  be  applied  to  these  tasks: 
heuristic  classification  and  heuristic  construction.  Heuristic 
classification  involves  the  identification  of  classes  that  can  be  defined 
by  a set  of  attributes  and  into  which  items  can  be  categorized  by  their 
goodness  of  match  to  each  class'  attributes.  It  is  used  for  diagnosis, 
repair,  interpretation,  and  selection.  Heuristic  construction  is  used  for 


Page  15 


synthetic  tasks,  such  as  design,  configuration,  scheduling,  and  planning. 
In  heuristic  construction,  the  problem  solver  utilizes  building  blocks, 
i.e.,  problem  components  or  partial  solutions,  and  either  incrementally 
builds  a solution  or  constructs  a complete  solution  that  then  requires 
refinement  until  the  solution  is  sufficient. 

Chandrasekaran  (1985)  describes  six  generic  problem-solving  tasks  in 
knowledge-based  reasoning,  including  classification,  state  abstraction, 
knowledge-directed  retrieval,  object  synthesis  by  plan  selection  and 
refinement,  hypothesis  matching,  and  assembly  of  compound  hypotheses  for 
abduction.  These  tasks  correspond  to  problem  solving  methods  that  can  be 
combined  to  perform  knowledge-based  reasoning  for  an  application.  Bylander 
and  Chandrasekaran  (1987)  suggest  that  generic  tasks  can  be  associated  with 
a specific  knowledge  acquisition  methodology. 

Boose  and  Bradshaw  (1987)  proposed  a set  of  knowledge  acquisition 
strategies  that  link  with  appropriate  problem-solving  methods.  The 
knowledge  acquisition  strategies  are  establishing  distinctions  between 
alternatives,  problem  decomposition,  combining  and  propagating  information, 
testing  of  knowledge,  combining  multiple  sources  of  knowledge,  incremental 
expansion  of  knowledge,  and  providing  process  guidance.  Kit  to  and  Boose 
(1990)  developed  the  Dialog  Manager  subsystem  for  the  AQUINAS  system  to 
select  knowledge  acquisition  strategies  for  particular  application  tasks. 
Dialog  manager  is  only  capable  of  recommending  methods  associated  with 
heuristic  construction  or  heuristic  classification  problem-solving  tasks, 
both  of  which  involve  primarily  problem  decomposition  and  establishing 
distinctions  as  knowledge  acquisition  strategies. 

Kitto  and  Boose  caution  that  a knowledge  acquisition  system  must  be 
able  to  acquire  knowledge  about  aspects  of  the  problem-solving  process  that 
are  unique  to  a given  application.  For  example,  Jacobson  and  Freiling 
(1988)  identified  specific  knowledge  requirements  for  troubleshooting  tasks 
in  addition  to  the  requirements  for  the  more  general  task  of  differential 
diagnosis,  such  as  test  set-up  and  measurement,  strategies  to  control  test 
selection,  and  interpretations  relating  test  results  to  diagnostic  beliefs. 
Kitto  and  Boose  report  that  for  electromechanical  diagnosis  information 
about  component  failure  rates,  test  times,  test  costs,  test  utility  and 
test  dependencies  are  required. 

McDermott  (1988)  presented  a preliminary  taxonomy  of  problem-solving 
methods  based  on  the  assumption  that  there  are  families  of  tasks  that  share 
abstract  control  knowledge  and  the  abstract  control  knowledge  provides 
strong  guidance  as  to  what  knowledge  is  required  and  how  that  knowledge 
should  be  encoded.  He  refers  to  the  control  methods  as  role-limiting 
methods  because  they  strongly  guide  knowledge  collection  and  encoding, 
i.e.,  task  specific  knowledge  fills  one  of  a few  number  of  roles  within  the 
control  framework.  McDermott  argues  that  it  is  likely  there  are  hundreds 
of  role-limiting  methods.  He  discusses  the  following  role-limiting  methods 
and  knowledge  acquisition  tools; 

1.  cover  and  differentiate  - implemented  in  MOLE 

2.  propose  and  refine  - implemented  in  SALT 


3.  qualitative  reasoning  - implemented  in  YAKA 

4.  acquire  and  present  - implemented  in  KNACK 

5.  extrapolate  from  a similar  case  - implemented  in  SIZZLE 

Finally,  Garg-Janardan  and  Salvendy  (1987)  describe  a conceptual 
framework  for  knowledge  elicitation  that  synthesizes  Newell  and  Simon's 
(1972)  theory  of  problem  solving  with  Kelly's  (1955)  theory  of  personal 
constructs.  Their  framework  suggests  that  the  invariant  and  variant  parts 
of  the  problem  must  be  identified  by  the  expert  during  knowledge 
acquisition.  The  invariant  parts  of  the  problem  space  can  be  defined  by  a 
set  of  characteristics,  reflecting  the  nature  of  the  components  of  the 
problem,  the  problem-solving  process,  and  the  adequacy  of  a problem 
solution.  The  variant  part  of  the  problem  consists  of  a set  of  states, 
initial  states,  intermediate  states,  and  final  states,  that  cannot  be  fully 
enumerated.  However,  the  expert  should  describe  a set  of  final  states  and 
how  each  of  these  states  may  be  reached.  The  authors  suggest  using  the 
theory  of  personal  constructs  to  elicit  information  about  the  invariant  and 
variant  portions  of  the  problem. 


2.3  The  Goal  Of  The  Current  Study 


What  researchers  in  psychology  and  artificial  intelligence  have 
neglected  to  specify  is  how  to  determine  in  the  first  place  whether  or  not 
an  application  task  involves  a certain  characteristic.  How  do  you  identify 
a diagnostic  task?  How  do  you  know  if  the  task  requires  declarative  or 
procedural  knowledge?  Or  both?  This  issue  is  not  entirely  self  evident 
but  is,  in  fact,  the  reason  that  knowledge  acquisition  remains  largely  an 
art.  Good  knowledge  engineers  have  little  difficulty  discerning  the 
fundamental  requirements  of  a task,  but  there  is  no  formal  method  for 
making  these  judgments  and  no  indication  of  how  accurate  they  are  once  the 
judgments  are  made.  The  current  study  explores  these  issues  and  attempts 
to  provide  a method  for  initiating  knowledge  acquisition  that  will 
elucidate  the  key  characteristics  of  a task  and  support  further  knowledge 
acquisition  activities. 
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3 THE  FIRST  INTERVIEW 


Initially,  the  research  was  to  entail  only  a single  interview  with 
each  of  the  experts  in  the  selected  fields.  However,  we  determined  early 
in  the  research  that  one  interview  would  not  be  sufficient  for  developing 
and  performing  even  minimal  verification  of  a taxonomy.  Thus,  it  was 
decided  that  two  interviews  would  be  performed:  one  to  provide  a means  of 
analyzing  the  problem  solving  task  and  generating  an  initial 
classification,  and  one  to  apply  the  recommendations  made  based  on  the 
first  interview  and  observe  the  results.  Thus,  the  primary  goal  of  the 
first  interview  was  to  acquire  enough  knowledge  about  each  problem  solving 
task  to  be  able  to  characterize  it  in  some  way  so  that  a taxonomy  could  be 
developed  to  direct  the  second  interview.  The  second  interveiw  would  then 
be  used  to  informally  verify  this  taxonomy.  The  following  sections 
describe  the  approach  used  to  develop  the  first  interview,  perform  the 
first  interview,  and  how  the  interview  was  performed. 


3.1  Developing  The  First  Interview 


In  developing  the  first  interview,  three  major  aspects  had  to  be 
addressed.  First,  a means  by  which  each  task  could  be  characterized  and 
rated  had  to  be  developed  so  that  a comparison  could  be  made  between  tasks. 
Second,  a set  of  problem  solving  tasks  had  to  be  selected  for  analysis 
using  this  method  of  characterization.  Third,  a knowledge  acquisition 
approach  had  to  be  identified/developed  that  could  be  used  to  acquire  the 
information  on  each  problem  solving  task.  Each  of  these  aspects  of  the 
approach  is  addressed  in  turn  below. 


3.1.1  Generating  A Set  Of  Task  Characteristics  - 

As  discussed  in  Section  2,  a literature  search  was  performed  in  both 
the  psychology  and  artificial  intelligence  fields  to  identify  existing 
approaches  to  problem  solving  taxonomies.  This  search  helped  to  identify  a 
list  of  123  characteristics  that  fell  into  15  different  categories  on  which 
a particular  problem  solving  task  could  be  rated.  The  orignial  list  of  123 
characteristics  is  provided  in  Appendix  A,  along  with  a discussion  of  the 
characteristics  by  category.  This  list  was  intended  to  be  relatively 
exhaustive  by  including  all  the  attributes  that  might  be  relevant  to  how 
problem  solving  tasks  could  be  categorized.  We  suspected  that  some 
redundancy  existed  and  that  some  of  the  characteristics  would  not  prove  to 
be  relevant  for  the  problem  of  knowledge  acquisition.  However,  we  wanted 
to  be  sure  that  as  much  as  possible  was  considered  when  analyzing  a problem 
solving  task. 

The  list  of  characteristics  compiled  for  this  study  originated  from  a 
compilation  of  existing  taxonomies  (Gawron,  Drury,  Czaja,  and  Wilkins, 
1989).  From  this  compilation,  we  generated  a list  of  characteristics  which 
fell  into  15  categories: 
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1.  Reasoning  Techniques  - characteristics  related  to  the  manipulation  of 
information  during  task  performance 

2.  Problem-Solving  Techniques  - search  strategies  for  finding  a solution 

3.  Inputs  - data  input  to  the  problem  solving  process 

4.  Task  Complexity  - characteristics  of  the  data  array  and  problem  solving 
space  that  influence  the  task 

5.  Technical  Dimension  - the  degree  to  which  the  task  requires  a technical 
background  or  skill 

6.  Motor  Processes  - physical  requirements  of  the  task 

7.  Information  Processing  - characteristics  of  the  way  data  is  manipulated 

8.  Problem  Solving  Tasks  - cognitive  operations  on  the  data 

9.  Recall  - the  role  of  memory  in  the  task 

10.  Perceptual  Processes  - the  perceptual  requirements  of  the  task 

11.  Environment  - physical  and  psychological  characteristics  of  the 
environment  "in  which  the  task  is  performed 

12.  Personal  Characteristics  - characteristics  required  to  perform  the  task 

13.  Type  of  Domain  - knowledge- rich  and/or  high  performance 

14.  Hardware/Equipment/Tools  - items  or  devices  used  during  the  task 

15.  Communication  Processes  - verbal  and  nonverbal  communication  during 
task  performance 

We  selected  these  categories  and  characteristics  because  of  their 
relevance  to  knowledge  acquisition  and  system  development.  We  felt  that 
some  of  the  characteristics  would  be  related  to  how  knowledge  acquisition 
should  be  conducted  and  others  would  be  more  related  to  how  an  intelligent 
system  for  the  task  should  be  designed. 


3.1.2  Selecting  A Set  Of  Problem  Solving  Tasks  - 

In  selecting  a set  of  problem  solving  tasks  for  analysis  in  this 
research,  the  goal  was  to  find  a representative  set  with  as  much  the 
variety  as  was  possible.  Initially,  only  10-12  tasks  were  to  be  analyzed, 
but  it  became  apparent  that  15-16  would  be  necessary  to  obtain  sufficient 
variety.  The  following  is  a list  of  the  tasks/jobs  identified  for  a 
preliminary  analysis: 


I 


w 
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o maintain  systems/diagnostics 
o repair  systems/making  the  fix 
o communications  monitoring/analysis 
o photo  analysis 
o training  pilots 
o training  astronauts 

o training  air  crews  (groups  to  work  together) 
o training  instructors 
o air  traffic  control 
o console  operations 
o weather  forecasting/modelling 
o program  management 
o facilities  management 
o cargo  loading 

o operating  a piece  of  equipment 
o software  design 
o software  maintenance/debugging 
o form  fill-out 
o training  reading 
o training  foreign  language  ' 
o oil  accident  investigation 
o prospecting 
o power  grid  management 
o computer  configuration 
o chemical  analysis 
o personnel  management 
o customer  service 


o 


sales 


o weapons  director/air  intercept 
o use  RMS/other  remote  manipulator 
o leadership  training 
o surgery 

o medical  diagnosis 
o accounting 
o marketing/advertising 
o knowledge  engineering 
o scientific  protocol  design 

This  list  was  generated  based  on  discussions  between  SwRI  and  the  sponsor. 
These  problem  solving  tasks  cover  a broad  range  of  expertise.  The  goal  was 
to  identify  as  many  different  areas  as  possible  initially,  and  then  to 
narrow  the  set  down  both  in  number  and  specificity,  so  that  human  experts 
could  be  identified  and  interviews  could  take  place  within  the  time  and 
budget  restrictions. 

In  order  to  begin  developing  some  mechanism  for  categorizing  tasks  so 
that  we  could  make  a selection  based  on  variability  between  tasks,  we  went 
through  each  of  the  37  initial  candidate  tasks  and  checked  the 
characteristics  out  of  the  123  we  listed  that  were  relevant.  The  matrix 
that  was  generated  appears  in  Appendix  F.  Based  on  this  preliminary 
analysis,  a set  of  distinctive  attributes  and  task  types  became  apparent. 
The  following  list  provides  the  task  types  that  we  felt  needed  to  be 
represented  in  the  set  selected  for  interview. 

1.  diagnostic  task 

2.  training  task 

3.  high  performance  task 

4.  form  fill-out 

5.  people-oriented  (soft)  task 

6.  design  task 

7.  planning  task 

8.  monitoring  with  a time  factor 
perceptually-oriented  task 


9. 
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10.  bin-packing/np-complete  task  (exponential/combinatorial  growth  problem) 

11.  numerical  task 

12.  data  intensive  (no  real  time  factor),  ie.  acquire  and  present 

As  a result,  we  interviewed  experts  in  the  following  areas: 

1.  (diagnostics)  medical  diagnosis 

2.  (diagnostics)  equipment  diagnosis 

3.  (training)  training  pilots 

4.  (training)  training  a foreign  language 

5.  (high  performance)  flight  controller  console  operations 

6.  (high  performance/KR)  surgery 

7.  (form  fill-out)  contracting 

8.  (people-oriented)  personnel  management /leadership  training 

9.  (design)  sofware  design 

10.  (planning)  acquisition  program  management 

11.  (monitoring/time)  air  traffic  control 

12.  (perceptual)  weather  forecasting 

13.  (bin-packing)  cargo  loading 

14.  (numerical)  accounting 

15.  (data  intensive/no  time)  DRAIR  generation 

16.  (planning)  scientific  protocol  design 

Obviously,  these  tasks  were  still  rather  broad  and  we  did  not  intend  to  try 
to  perform  knowledge  acquisition  on  the  entire  area  indicated  by  certain 
labels.  Instead,  we  planned  on  allowing  the  experts'  specific  job 
requirements  to  narrow  the  scope  in  each  problem  solving  area.  Individuals 
considered  experts  in  the  selected  areas  were  identified  and  asked  to  take 
part  in  the  effort.  Some  of  the  experts  were  available  through  the  Air 
Force,  while  others  were  available  through  SwRI  contacts.  Most  were  found 
in  the  San  Antonio  area  so  as  to  minimize  the  time  and  expense  required  for 
travel.  A description  of  each  of  the  tasks,  along  with  the  source  of 
expertise  and  issues  concerning  the  selection  of  an  approach  for  the  second 
interview,  is  provided  in  Appendix  B,  the  Technical  Journal. 
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3.1.3  Content  Of  The  First  Interview  - 

The  goal  of  the  first  interview  was  to  acquire  sufficient  information 
about  the  problem  solving  task  to  rate  each  task  on  a scale  of  one  to  four 
in  each  of  the  123  task  characteristics.  However,  the  interview  could  not 
simply  consist  of  asking  the  expert  if  he/she  felt  that  the  task  had 

certain  attributes.  For  example,  we  could  not  ask  whether  they  thought 

that  they  used  any  spatial  reasoning  or  induction.  As  a result,  we  had  to 
develop  an  interview  that  would  allow  us  to  determine,  based  on  the 

expert's  responses,  to  what  extent  a particular  task  involved  a particular 
characteristic.  For  example,  to  evaluate  the  inputs  to  the  task,  we 

constructed  the  following  question: 

What  types  of  inputs  to  the  problem  do  you  have?  How  do  you  recognize  the 
problem?  Do  you: 

- see  something 

- hear  something 
talk  to  someone 
feel  something 

- read  something  — book,  test  equipment,  computer  display 

- detect  it  based  on  past  data? 

These  questions  provided  a guide  to  ensure  that  we  obtained  information 
about  each  of  the  123  characteristics  during  the  interview.  We  framed  the 
questions  such  that  they  were  understandable  to  a lay  person;  we  attempted 
not  to  use  any  jargon. 

Because  the  first  interview  was  done  with  little  to  no  knowledge  of 
the  task,  the  interview  had  to  be  designed  so  that  it  would  be  appropriate 
for  any  task  type.  Thus,  to  design  the  interview,  we  examined  the 
literature  on  problem  solving  in  order  to  identify  a standard  format  for 
how  problems  are  solved.  The  standard  structure  to  problem  solving  comes 
from  the  field  of  "heuristic."  Heuristic  (or  heuretic)  is  the  study  of  the 
methods  and  rules  of  discovery  and  invention.  Webster's  dictionary  defines 
it  as  involving  or  serving  as  an  aid  to  learning,  discovery,  or  problem 
solving  by  experimental  trial  and  error  methods.  The  process  of  problem 
solving  can  be  subdivided  into  four  major  steps  (Polya,  1957): 

1.  Understanding  the  problem 

2.  Devising  a plan 

3.  Carrying  out  the  plan 

4.  Looking  back/assessing  the  results 

These  steps  provide  a process  by  which  all  problems  can  be  solved.  As  a 
result,  we  used  these  steps  as  a way  of  organizing  and  classifying  the 
questions  that  we  wished  to  ask  the  experts.  The  questions  centered  on 
obtaining  the  information  we  needed  to  determine  how  an  expert  performs 
each  of  the  four  steps  in  problem  solving.  Such  information  would  then  be 
useful  in  determining  how  to  rate  each  of  the  task  characteristics  with 
respect  to  a problem  solving  task.  The  result  was  a questionnaire 
consisting  of  67  questions.  The  original  questionnaire  is  given  in 

Appendix  D.  Not  all  of  these  questions  needed  to  be  asked  in  every 
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interview.  Some  depended  on  the  answers  to  other  questions.  The  primary 
questions  are  marked  with  an  asterisk. 


3.2  Performing  The  First  Interview 


The  length  of  the  questionnaire  in  the  initial  interview  was  rather 
intimidating.  In  order  to  put  the  expert  at  ease  from  the  start,  we  made 
sure  that  he/she  understood  what  was  stated  at  the  top  of  the 
questionnaire: 

’’There  are  no  right  answers  or  good  answers  to  these  questions.  Some 
of  the  questions  may  be  phrased  so  that  you  believe  one  or  another 
response  is  appropriate  or  somehow  better  than  another.  For  example, 
it  may  sound  good  to  say  you  plan  up  front  all  your  actions  in  solving 
a problem  or  doing  the  job  even  if  some  of  those  actions  occur 
somewhat  automatically.  Please  do  not  fall  into  this  bias.  For  this 
example,  it  is  good  to  plan  but  it  is  also  good  not  to  plan  too  much 
and  to  have  some  actions  that  occur  automatically.  We  want  to  assess 
all  aspects  of  your  job  with  as  much  fidelity  as  possible.  What 
results  from  these  interviews  will  help  us  to  better  understand  the 
unique  aspects  of  your  work  and  how  we  could  learn  more  about  it  in 
the  future  so  we  can  design  computer  systems  that  help  you  perform 
your  job  more  easily. 

Also,  we  want  to  assure  you  that  under  no  circumstances  will  your 
answers  go  to  your  supervisor  or  anyone  else  outside  of  this  room. 
You  are  responding  to  this  questionnaire  completely  anonomyously . 
Because  of  this,  we  want  you  to  provide  the  operational  answers  to 
these  questions,  not  the  official  ones.  We  want  to  know  what  you 
really  do  on  your  job." 

These  instructions  attempted  to  win  the  confidence  of  the  expert  and  to 
provide  a sense  of  anonymity.  We  did  not  read  these  instructions  verbatim 
at  each  interview,  instead  on  the  first  contact  over  the  phone,  we 
discussed  the  research  project  and  paraphrased  these  instructions.  All  of 
the  experts  seemed  very  comfortable  in  the  interview  and  seemed  to  attempt 
to  present  a completely  faithful  description  of  their  tasks. 

We  conducted  the  initial  interview  in  an  open  question/answer  format. 
We  attempted  to  keep  the  interview  as  flexible  as  possible,  allowing  the 
expert  to  continue  to  discuss  an  issue  until  he/she  had  exhausted  that  line 
of  inquiry.  At  those  points,  we  would  ask  another  question.  Our  only  goal 
was  to  have  answers  for  each  of  the  questions  whether  we  had  directly  asked 
the  question  or  the  expert  had  volunteered  the  answer  during  the  course  of 
discussion. 

We  conducted  the  interview  at  the  expert's  workplace  — either  in 
their  office  or  a nearby  conference  room.  The  experts  had  access  to  the 
material  they  generally  use  in  the  course  of  their  jobs  and  could  refer  to 
the  material  if  necessary.  In  a couple  of  cases,  we  obtained  a brief  tour 
of  the  facility  or  lab  where  the  task  was  performed. 
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In  the  initial  interview,  we  collected  audiotapes  of  the  interview  and 
took  notes.  We  found  the  audiotapes  were  not  useful  to  us  later  in 
characterizing  the  task  because  we  were  able  to  acquire  the  detail  that  we 
needed  from  our  handwritten  notes.  Furthermore,  the  quality  of  the  tapes 
were  not  always  very  good.  The  lack  of  use  of  audio  tapes  is  not  a general 
conclusion  for  knowledge  acquisition,  rather  we  found  they  were  not  useful 
for  this  research  project.  Where  extensive  and  accurate  detail  is  needed, 
we  suspect  that  audio  tapes  are  quite  useful. 
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4 EVALUATING  THE  FIRST  INTERVIEW 


The  major  goal  in  evaluating  the  results  of  the  first  interview  was  to 
develop  a principled  way  of  classifying  tasks  based  on  the  knowledge 
acquired  from  the  first  interview  and  to  make  recommendations  concerning 
how  to  proceed  with  the  knowledge  aquisition  process  based  on  this 
classification.  The  data  generated  from  the  first  set  of  interviews 
consisted  of  a 16  by  123  matrix  with  a rating  of  zero  to  four  for  each  of 
the  123  characteristics  for  each  of  the  16  tasks.  Rating  of  the 

characteristics  for  each  task  was  performed  as  follows: 

o Two  raters  reached  agreement  on  the  ratings  for  each  task  on  the  123 
attributes.  The  raters  used  a 5 point  Likert-type  scale  in  which  0 
indicated  the  attribute  is  not  relevant  at  all  for  the  task  up  to  4 
indicating  that  the  attribute  is  critical  for  task  performance. 

o The  ratings  were  used  as  a measure  of  the  importance  of  a 
characteristic  for  task  performance.  Initially,  before  the  first 
interview,  we  simply  checked  whether  or  not  the  characteristic  was 

involved  in  task  performance.  That  approach  allowed  no  discrimination 
between  central  and  peripheral  characteristics  of  a task.  We  adopted 
the  5 point  scale  to  provide  greater  discrimination. 

o We  assigned  the  ratings  for  each  task  after  the  inital  interview  and 
again  after  the  second  interview.  We  considered  each  characteristic  in 
relation  to  a task  and  reached  agreement  on  a value  based  on  the 

information  we  acquired  in  the  interview.  Rating  each  task  on  the  123 
characteristics  required  approximately  30  minute.s. 

The  resulting  matrix  is  provided  in  Appendix  G.  We  wanted  to  answer  a 

number  of  questions  concerning  the  data  in  the  matrix,  including  what 
characteristics  are  relevant  for  a meaningful  classification  of  the  tasks, 
which  tasks  are  related  based  on  their  characteristics  and  which  were  not, 
and  what  the  key  characteristics  are  about  a class  of  tasks  that  would 
affect  the  way  we  would  want  to  proceed  with  the  second  interview. 

Based  on  the  first  interview,  we  had  a good,  general  sense  of  what 
each  task  involved.  We  also  felt  comfortable  in  making  recommendations 
concerning  how  an  intelligent  tutoring  system  should  be  developed  and  how 
the  subsequent  interviews  should  proceed.  However,  the  ideas  on  how  we 
would  approach  the  rest  of  the  knowledge  acquisition  process  for  a given 
task  was  highly  dependent  on  our  experience  in  performing  knowledge 
acquisition.  We  found  it  difficult  to  verbalize  what  made  us  decide  to 
choose  one  direction  over  another.  The  approach  was  more  a "gut  feel," 
which  was  exactly  what  we  were  trying  to  avoid.  As  a result,  we  analyzed 
the  data  in  a number  of  formal  ways,  including  simple  permutations  of  the 
data  such  as  orderings  and  comparisons  and  statistical  analyses  such  as 
cluster  and  factor  analysis.  These  analyses  are  described  below. 
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4.1  A Preliminary  Look  At  The  Data 


A preliminary  look  at  the  data  involved  an  informal  approach  to 
manipulating  the  data.  The  ratings  for  each  task  on  the  123  task 
characteristics  vere  entered  into  a database  and  various  analyses  vere 
performed,  including  pairwise  comparisons,  orderings,  and  counts.  The 
following  summarizes  these  informal  analyses  and  the  results.  The  data 
reports  appear  in  Appendix  I. 


4.1.1  Pairwise  Comparison  - 

The  pairwise  comparison  of  the  data  generated  a count  of  the  number  of 
equal  ratings  for  each  possible  pairing  of  the  16  tasks.  Thus,  for 
example,  the  rating  given  to  each  of  the  123  characteristics  was  compared 
for  pilot  training  and  accounting  and  a count  generated  for  how  many  were 
exactly  equal  and  how  many  were  either  equal  or  different  by  only  one  point 
on  the  rating  scale.  The  pairwise  comparison  for  pilot  training  and 
accounting  resulted  in  a count  of  29  out  of  123  characteristics  that  were 
rated  identically,  and  a count  of  53  of  the  123  characteristics  that  were 
rated  similarly.  Of  all  of  the  124  pairs  generated,  the  minimum  count  of 
identical  ratings  was  28  and  the  maximum  count  was  72.  The  minimum  count 
of  similar  ratings  was  51,  while  the  maximum  count  was  101.  Using  these 
pairwise  comparisons,  the  data  was  printed  ordered  by  least  to  most  equal 
and  least  to  most  similar.  These  reports  appear  in  Appendix  I. 

Based  on  the  orderings,  it  is  possible  to  quickly  identify  tasks  that 
appear  to  be  more  and  less  similar  according  to  the  ratings  they  received 
on  the  123  task  characteristics.  For  example,  it  is  apparent  that  pilot 
training  is  very  dissimilar  to  almost  all  other  tasks  examined  because  it 
appears  in  nine  of  the  first  ten  pairs  in  the  least  similar  ordering.  Air 
traffic  control,  surgery,  and  console  operations  also  appear  consistently 
as  being  least  similar  to  most  other  tasks.  These  tasks  seem  to  generate 
individual  clusterings,  possibly  based  on  the  fact  that  they  all  had  a 
physical  component  to  performance  of  the  task.  On  the  other  hand,  tasks 
such  as  form  fill-out,  leadership  training,  accounting,  program  management, 
DRAIR  generation,  software  design,  and  protocol  design  constitute  the 
majority  of  the  pairings  in  the  twenty-five  pairs  that  were  most  similar 
(the  last  25  pairings  in  the  similar  listing).  Based  on  this  analysis,  one 
could  conclude  that  these  seven  tasks  clustered  due  to  some  common  set  of 
attributes,  possibly  related  to  issues  in  communications  and  lack  of 
procedural  knowledge  to  direct  the  progress  of  task  performance.  The 
remaining  tasks,  such  as  cargo  loading,  weather  prediction,  equipment 
diagnosis,  medical  diagnosis,  and  language  training  appear  to  be  scattered 
around  among  the  most  and  least  similar  tasks.  It  is  unclear  from  this 
minimal  analysis  to  what  this  scattering  may  be  attributed. 


4.1.2  Ordered  Characteristic  Count  - 

Another  analysis  performed  on  the  data  involved  examining  the  ratings 
with  respect  to  the  characteristics  themselves.  This  analysis  indicated 
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which  characteristics  were  discriminating.  Those  characteristics  that 
receive  similar  ratings  for  all  16  tasks  do  not  discriminate  between  those 
task.  Those  that  are  maximally  different  between  tasks  are  the  most 
discriminating. 

A count  was  generated  for  each  characteristic  for  each  of  the  five 
possible  ratings.  That  is,  the  number  of  tasks  receiving  a rating  of  0,  of 
1,  of  2,  of  3,  and  of  4 was  provided  for  each  characteristic.  Thus,  for 
example,  the  characteristic  "statistical"  had  13  tasks  where  the  rating  was 
0,  two  tasks  where  the  rating  was  1,  one  task  where  the  rating  was  2,  zero 
tasks  where  the  rating  was  3,  and  zero  tasks  where  the  rating  was  4.  This 
count  was  then  printed  out  in  two  different  reports,  one  ordered  by  lowest 
to  highest  count  for  the  zero  ratings  and  one  ordered  by  lowest  to  highest 
count  for  the  four  ratings.  These  reports  appear  in  Appendix  I. 

Based  on  this  analysis,  several  characteristics  appear  to  be 
non-discriminating  because  they  have  a 0 rating  for  most  of  the  tasks. 
These  characteristics  are:  kinesthetic,  sleep,  lighting,  scanning  a 
display,  statistical  reasoning,  computerize,  tabulate,  vehicles,  reflex, 
weight,  acceleration,  complex  continuous  motor  responses,  contaminants, 
age,  compound  (actions),  electricity,  gender,  height,  code,  isolation, 
magnetism,  and  weapon  systems.  Only  a few  characteristics  seemed  to  be 
non-discrinimating  due  to  a high  count  in  four  ratings.  These  were  verbal 
(10),  temporal  (10),  and  procedural  (12).  Other  characteristics,  such  as 
branching,  dynamism,  cognitive  attentiveness,  instruments,  spatial,  visual, 
historical,  instrumentation,  plan,  direct,  supervise,  and  monitoring, 
appear  to  have  high  counts  in  the  0 and  4 ratings,  and  lesser  counts  in  the 
middle  ratings  of  1,  2,  and  3,  thus  forming  a concave  curve  with  their  task 
counts.  These  characteristics  should,  therefore,  provide  a means  of 
distinguishing  tasks. 


4.1.3  Important/Unimportant  Characteristics  By  Task  - 

A final  informal  data  analysis  examined  the  most  and  least  important 
characteristics  for  each  task.  A list  was  generated  for  each  task  that 
provided  the  characteristics  that  received  a zero  rating  and  a four  rating. 
This  provided  a means  of  seeing,  by  task,  which  were  key  attributes  and 
which  were  not.  These  lists  appear  in  Appendix  I.  In  addition,  the 
0-rated  and  4-rated  characteristics  are  listed  for  each  task  in  the 
Technical  Journal,  Appendix  B. 


4.2  Statistical  Analyses 


The  ratings  of  the  characteristics  entered  into  a factor  analysis  and 
cluster  analysis.  Some  characteristics  were  dropped  from  the  analysis 
because  the  ratings  for  all  16  tasks  was  0.  Additionally,  we  collapsed  the 
scores  across  a category  of  characteristics  if  a summary  score  for  that 
category  made  sense.  For  example,  a summary  score  for  the  category 
"inputs"  reflects  the  degree  to  which  the  task  requires  a lot  of  external 
information.  A summary  score  for  the  category  "reasoning  techniques"  would 
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not  produce  a meaningful  index. 


4.2.1  Factor  Analysis  Of  The  123  Task.  Characteristics  - 

An  inspection  of  the  eigenvalues  of  the  principle  components  factor 
analysis  lead  to  three  factors  being  retained.  The  three  factors  were 
rotated  with  a varimax  rotation  to  allow  for  correlated  factors. 

Factor  1 accounted  for  28%  of  the  variance.  This  factor  seems  to  be  a 
design  factor  with  characteristics  such  as  propose  and  refine,  decompose, 
communication,  and  a large  amount  of  information  recalled  loading  heavily 
on  the  factor.  Factor  2 accounted  for  182  of  the  variance.  This  factor 
indicates  an  underlying  dimension  in  the  questionnaire  corresponding  to 
tasks  in  which  a physically-based  skill  plays  a large  role. 
Characteristics  such  as  spatial  reasoning,  deduction,  specialization,  a 
large  number  of  inputs,  high  complexity,  a large  perceptual  component,  an 
adverse  environment,  psychologically  stressful,  and  high  performance  domain 
had  high  loadings  on  this  factor.  Finally,  factor  3 accounted  for  12%  of 
the  variance  and  revealed  a factor  for  tasks  requiring  statistical 
reasoning,  modelling,  and  inductive  reasoning. 


4.2.2  Cluster  Analysis  Of  The  16  Tasks  - 

An  eight  cluster  solution  revealed  the  following  task  clusters: 

1.  software  design  and  protocol  design 

2.  pilot  training  and  surgery 

3.  cargo  loading,  accounting,  program  management,  leadership  training, 
equipment  diagnosis,  drair  generation,  form  fill-out 

4.  medical  diagnosis 

5.  teaching  a foreign  language 

6.  air  traffic  control 

7.  weather  forecasting 

8.  console  operations 


Software  design  and  protocol  design  clustered  because  of  their 
similarity  on  the  following  characteristics:  temporal  reasoning,  forming 
sub-goals,  generate  and  test  problem  solving,  propose  and  refine  problem 
solving,  acquire  and  present,  decomposition  of  the  problem,  and  their 
perceptual,  environmental,  psychological,  and  physical  requirements.  These 
problems  are  synthetic  problems,  requiring  the  problem  solver  to  decompose 
the  problem,  synthesize  a solution,  and  present  the  results.  They  do  not 
involve  perceptual  skills,  are  not  performed  in  an  adverse  environment,  are 
not  stressful  and  do  not  require  specific  physical  characteristics. 
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Pilot  training  and  surgery  clustered  because  they  correspond  on  the 
following  characteristics:  spatial  reasoning,  temporal  reasoning, 
modelling,  forming  sub-goals,  use  of  a formula/procedure/algorithm, 
specialization,  psychological  stress,  large  amount  of  information  recalled 
during  task  performance,  and  the  overall  task  has  both  knowledge  rich  and 
high  performance  components.  Although  on  the  surface  pilot  training  and 
surgery  may  appear  to  be  very  different,  they  share  many  fundamental 
characteristics.  The  problem  solver  must  reason  about  spatial  and  temporal 
information.  They  form  sub-goals  and  often  use  a formula  or  procedure  to 
accomplish  a sub-goal.  Finally,  they  are  both  stressful. 

The  third  cluster  emerged  because  of  correspondence  on  the  following 
characteristics:  forming  sub-goals,  acquire  and  present  problem  solving 
strategy,  no  perceptual,  physical,  or  psychological  requirement,  and  they 
are  all  knowledge  rich  tasks.  This  cluster  relates  to  the  differentiation 
that  has  emerged  from  this  study  concerning  the  type  of  training  received 
to  become  proficient  at  the  task.  This  differentiation  will  be  discussed 
in  detail  in  Sections  4.5  and  4.6. 


4.2.3  Summary  - 

The  factor  analysis  revealed  three  main  factors  in  the  questionnaire. 
The  first  factor  consisted  of  characteristics  related  to  design  tasks,  the 
second  factor  consisted  of  characteristics  of  tasks  involving  a 
physically-based  skill,  and  the  third  factor  corresponded  to 
characteristics  of  tasks  involving  statistical  reasoning  and  mental 
modelling.  These  factors  were  evident  in  the  task  clustering.  The  first 
cluster,  software  design  and  protocol  design,  includes  the  design  tasks  we 
examined.  The  second  cluster,  pilot  training  and  surgery,  are  tasks  that 
require  physically-based  skills.  The  relevance  of  the  third  factor  is  less 
clear.  Protocol  design,  medical  diagnosis,  weather  forecasting,  and  drair 
generation  require  statistical  reasoning  while  software  design,  weather 
forecasting,  console  operations,  surgery  and  medical  diagnosis  require  the 
use  of  mental  models.  This  factor  does  not  clearly  map  to  the  cluster 
analysis. 


4.3  An  Analysis  Based  On  Experience 


Once  all  of  the  data  was  analyzed,  both  informally  and  formally,  to 
see  if  any  insight  could  be  gained  into  the  relationships  among  tasks  and 
characteristics,  we  still  had  difficulty  developing  an  interpretation  of 
the  data  that  could  provide  some  direction  to  the  research.  As  a result, 
we  spent  some  time  brainstorming  based  on  our  own  experience  in  knowledge 
acquisition,  describing  what  we  felt  would  be  a reasonable  approach  to  the 
second  interview  for  each  task  and  why.  We  also  discussed  how  we  imagined 
a tutoring  system  for  such  a task  should  be  designed  and  why.  Initially, 
much  of  what  was  generated  was  based  on  ,fgut  feel"  or  intuition,  which  was 
exactly  what  we  were  trying  to  avoid  in  this  research.  However,  after  a 
while  some  patterns  began  to  emerge  that  determined  how  we  should  proceed 
with  each  of  the  sixteen  tasks  and  why.  Many  of  the  rules  of  thumb  for 
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preceding  with  the  second  interview  are  listed  at  the  beginning  of  the 
notes  in  the  Technical  Journal,  Appendix  B.  However,  some  other  more 
general  patterns  also  emerged  that  revolved  around  whether  or  not  the  task, 
was  formally  trained. 

The  issue  of  whether  or  not  a task  is  formally  trained  was  not  clearly 
delineated  in  the  characteristics  by  which  we  were  rating  each  task. 
However,  it  was  clear  from  our  initial  interview  the  process  of  becoming  an 
expert  differed  between  tasks,  and  we  found  ourselves  recommending  an 
examination  of  the  curriculum  for  the  second  interview  for  any  task  that 
was  formally  trained  in  the  operational  environment.  As  a result,  we 
categorized  the  tasks  by  whether  or  not  an  individual  received  formal 
training  in  performing  the  actual  task(s)  that  constituted  their  area  of 
expertise.  The  tasks  fell  into  two  relatively  equal  sets  where  aircraft 
piloting,  air  traffic  control,  console  operations,  weather  forecasting, 
cargo  loading,  foreign  language,  surgery,  and  medical  diagnosis  all  are 
heavily  trained  before  an  individual  is  allowed  to  perform  the  task,  and 
where  equipment  diagnosis,  form  fill-out,  program  management,  DRAIR 
generation,  software  design,  leadership,  accounting,  and  protocol  design 
are  not  formally  trained.  Tasks  in  the  former  set  may  or  may  not  require 
any  general  education  beyond  high  school,  but  they  are  extensively  trained, 
often  to  the  point  where  the  problem  solving  methodology  for  the  task  has 
become  highly  proceduralized.  Tasks  appearing  in  the  latter  set  tend  to 
require  at  least  a general  background  education  in  a particular  field  such 
as  business,  computer  science  operations  research  as  a foundation  on  which 
they  can  build  expert  skill  through  experience  on  the  job. 

Once  these  two  sets  of  tasks  were  generated,  we  then  examined  the 
other  characteristics  of  the  tasks  to  see  if  there  were  any  unifying 
themes.  Among  the  formally  trained  tasks,  the  unifying  characteristics 
seemed  to  be  an  issue  involving  human  safety,  as  well  as  quite  often  a 
physical  component.  They  were  also  highly  proceduralized,  even  if  the 
experts  did  not  necessarily  follow  a procedure  once  they  became  expert. 
Individuals  entering  the  field  are  taught  a procedure  to  follow  that  allows 
them  to  become  efficient  problem  solvers  in  the  domain.  Tasks  that  were 
not  formally  trained  tended  to  be  less  well-defined  in  terms  of  goals  and 
results.  Such  tasks  did  not  tend  to  have  definitive  right  and  wrong 
answers  and  problem  solving  tended  to  be  oriented  towards  breaking  the  task 
down  into  relatively  independent  subtasks.  This  approach  helped  to  deal 
with  the  complexity  and  inexactness  of  the  problem. 

In  addition  in  those  tasks  that  involved  a procedure,  the  skills 
required  for  the  task  tended  to  build  sequentially  on  one  another.  So  for 
example,  when  a person  learns  to  fly  a plane  they  begin  by  learning  how  to 
fly  level,  then  to  perform  simple  maneuvers,  then  to  take-off,  then  to 
land.  Each  skill  builds  on  the  previous  skill.  However,  the  skills 
required  in  the  set  of  tasks  that  are  not  formally  trained  tended  to  be 
componential.  That  is,  the  problem  solver  has  a toolbox  of  skills  relevant 
for  different  aspects  of  the  problem  solving  process.  They  learn  each  of 
the  skills  relatively  independently.  For  example,  the  expert  in  protocol 
analysis  had  skill  in  experimental  design,  statistical  analysis,  research 
methodology,  and  electronics.  These  are  independent  skills  and  are  learned 
during  a formal  education  in  a content  areas,  such  as  biomedical 
engineering.  Based  on  these  observations,  we  labeled  the  first  set  of 
tasks  sequential  tasks  and  the  second  set  we  call  componential  tasks. 
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4.4  Results  And  Conclusions  From  The  First  Interview 


When  all  of  the  various  analyses  of  the  data  from  the  first  interview 
were  compared,  we  began  to  see  patterns  in  the  data  that  confirmed  our 
experiential  analysis.  For  example,  the  seven  cluster  statistical  analysis 
generated  the  following  clusters: 

1.  pilot  training,  surgery 

2.  software  design,  protocol  design,  cargo  loading,  accounting,  program 
management,  leadership,  equipment  diagnosis,  DRAIR  generation,  form 
fill-out 

3.  medical  diagnosis 

4.  foreign  language  training 

5.  air  traffic  control 

6.  weather  forecasting 

7.  console  operations 

The  list  of  tasks  in  the  second  cluster  corresponds  exactly,  except  for 
cargo  loading,  with  the  set  of  tasks  that  ware  not  formally  trained.  The 
remaining  seven  clusters,  which  almost  all  are  made  up  of  a single  task, 
correspond  to  the  set  of  tasks  that  are  formally  trained. 

The  eight  cluster  analysis  given  in  Section  4.2.2  corresponds  with  and 
reinforces  our  informal  analyses.  The  first  cluster,  software  design  and 
protocol  design,  and  the  large  third  cluster  primarily  contain  tasks  that 
do  not  receive  formal  training  in  the  operational  environment,  such  as 
program  management,  accounting,  and  drair  generation.  However,  these  tasks 
require  a lot  of  background  knowledge  often  obtained  through  formal 
education.  The  remaining  clusters  are  tasks  that  receive  a large  amount  of 
training  in  the  operational  environment.  Surgical  training  stems  from 
years  of  internships  and  residencies,  air  traffic  control  involves  training 
specific  to  each  airport  at  which  the  controller  works,  and  console 
operations  requires  both  a formal  educational  relevant  to  the  specific 
console  and  continual  on-the-job  training  to  remain  proficient. 

In  addition,  the  factor  analysis  of  the  characteristics  indicated  that 
the  clusterings  from  both  the  seven  and  eight  cluster  analyses  were  due  to 
differences  among  the  design,  high  performance,  and  model-based/statistical 
factors  that  roughly  correspond  to  the  attributes  we  saw  intuitively  as 
being  key  to  the  commonalities  that  were  repsonsible  for  the  generation  of 
the  two  sets  of  tasks.  The  information  gained  from  correlating  all  of 

these  various  analyses  was  used  to  begin  generating  a tentative  problem 
solving  taxonomy  to  support  the  knowledge  acquisition  process.  Figure  2 
illustrates  a proposed  taxonomy  as  a result  of  the  research.  At  the  point 
of  completing  the  analysis  of  the  first  interview,  we  had  generated  the 
first  layer  of  this  taxonomy,  namely  the  breakdown  into  sequential  vs. 
componential. 
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Figure  2.  The  proposed  problem  solving  taxonomy 
to  support  knowledge  acquisition 
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5 THE  SECOND  INTERVIEW 


As  a result  of  the  analysis  of  the  first  interview,  ve  generated  a 
particular  approach  to  each  of  the  sixteen  follow-up  interviews  that  we 
would  perform.  Further  details  on  the  approach  suggested  for  each  task, 
and  the  justifications,  are  provided  in  the  Technical  Journal,  Appendix  B. 
In  the  Technical  Journal,  we  also  attempted  to  document  many  of  the  rules 
of  thumb  that  seemed  to  be  driving  the  decision  to  use  an  example,  discuss 
a curriculum,  or  perform  a task  breakdown,  and  to  what  level  of  detail. 


5.1  Developing  The  Second  Interview 


Once  a clustering  of  the  tasks  became  apparent,  we  examined  the 
implications  for  the  second  interview  for  each  task.  However,  when  we 
compared  what  ve  felt  we  should  do  for  the  second  interview  based  on  our 
own  intuition,  versus  what  seemed  appropriate  based  on  the  clusterings,  ve 
saw  that  the  clusterings  did  not  have  as  much  impact  as  expected  on  the 
desired  approach.  Based  on  this  assessment,  ve  determined  that  the  second 
interview  is  still  too  early  in  the  knowledge  acquisition  process  for  much 
delineation  to  take  place  in  terms  of  how  to  proceed  with  the  knowledge 
acquisition  task.  For  example,  even  though  a wealth  of  information  is 
available  through  existing  curriculums  for  those  tasks  that  are  formally 
trained,  we  sometimes  felt  that  the  second  interview  was  too  early  to 
effectively  utilize  the  information.  In  most  cases,  we  believed  that  going 
through  an  example  in  some  form  would  be  the  most  appropriate  approach  to 
the  second  interview.  The  differences  among  tasks  was  exploited  mainly  in 
how  the  second  interview  should  elicit  the  example,  how  many  examples 
should  be  examined,  and  the  way  the  examples  should  be  framed  to  gain 
further  insight  into  the  task. 

The  major  exceptions  to  the  use  of  an  example  were  foreign  language 
training,  software  design,  program  management,  and  leadership  training. 
Foreign  language  training  was  an  exception  because  an  example  is  almost 
meaningless,  so  examination  of  a component  of  the  curriculum  was 
recommended.  Software  design  and  program  management  were  exceptions 
because  the  tasks  are  so  large  and  ill-defined  at  their  highest  level  that 
an  example  would  have  little  meaning  even  if  it  could  be  formulated.  Thus, 
we  selected  one  of  the  components  of  software  design  and  of  program 
management  on  which  to  focus  further  discussion.  This  would  allow  an 
iterative  and  principled  approach  to  breaking  the  task  down  into  sub-tasks 
until  a task  of  a workable  size  was  found.  Leadership  training  was  an 
exception  because  it  was  unclear  how  an  example  could  even  be  formulated 
from  which  a discussion  could  evolve.  Because  there  are  a number  of 
techniques  that  are  used  in  leadership  training  (TQM)  ve  chose  to  examine 
one  of  those  techniques,  namely  team  building. 

One  minor  exception  to  the  use  of  examples  in  the  second  interview 
occured  with  aircraft  pilot  training.  For  this  task,  the  desired  approach 
was  to  examine  an  example,  but  we  wanted  to  examine  the  easiest  examples, 
preferably  from  the  first  few  flights  that  a student  pilot  would  take.  In 
this  case,  a well-defined  curriculum  existed  and  it  was  used  to  guide  the 
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selection  of  the  examples  that  would  be  most  useful  in  futhering  our 
understanding  of  the  task.  Of  course,  this  is  how  they  teach  the  students 
to  perform  the  task,  as  well,  through  the  use  of  well-selected  examples. 

In  the  cases  where  the  second  interview  should  consist  of  examining  an 
example,  the  differences  among  tasks  was  exploited  mainly  in  how  the  second 
interview  should  elicit  the  example  and  how  many  examples  should  be 
examined  in  that  second  interview.  In  general,  if  the  task  is  data 

intensive,  then  a sequence  of  examples  that  built  on  increasing  data 
complexity  was  used.  In  cases  where  the  task  included  a strong  procedural 
component,  the  example  was  to  be  used  to  provide  structure,  but  general 
rules  were  expected  as  part  of  the  outcome  of  the  interview. 

As  a result  of  the  process  of  generating  recommendations  on  how  to 
approach  the  second  interview  for  each  task,  additional  levels  in  the 
problem  solving  taxonomy  were  becoming  apparent.  Issues  concerning 

1.  complex/data  intensive  vs.  less  complex/minimal  use  of  data 

2.  well-defined  begin  and  end  points  vs.  continuous  task 

3.  procedural  vs.  non-procedural 

were  used  to  help  determine  the  appropriate  approach  to  the  second 
interview.  These  levels  can  be  seen  in  the  taxonomy  provided  in  Figure  2. 


5.2  Performing  The  Second  Interview 

The  second  interview  was  performed  much  as  the  first.  After  we 
developed  the  approach  to  the  second  interview,  we  contacted  the  expert. 
If  we  wanted  the  expert  to  discuss  a specific  example  or  if  we  wanted  them 
to  describe  the  data  that  they  use  in  problem  solving,  we  asked  the  expert 
to  bring  the  appropriate  materials  to  the  interview.  This  material  and  our 
chosen  strategy  guided  the  interview  process.  We  had  much  less  control 
than  in  the  first  interview  because  the  example  or  relevant  data  were 
unknown  to  us  at  the  start.  However,  the  process  was  still  an  interview; 
we  asked  questions  freely  during  the  problem  solving  process.  We  also 
interjected  suggestions  concerning  how  to  simplify  or  complicate  an  example 
in  order  to  test  our  conception  of  what  caused  a task  to  become  more 
complicated. 

Again,  we  conducted  the  interview  at  the  expert's  workplace  and  they 
had  access  to  any  additional  material  they  needed.  In  many  cases,  we  were 
able  to  conduct  the  interview  in  the  environment  where  the  task  took  place, 
such  as  in  aircraft  cockpit  simulator,  an  operating  room,  or  the  weather 
map  room.  We  chose  not  to  audio  tape  any  of  the  interviews  because  we 
found  the  quality  of  the  tapes  to  be  low,  they  interrupted  the  flow  of  the 
interview,  and  we  did  not  need  them  later  for  our  purposes.  However,  we 
did  take  extensive  notes  and  based  our  analysis  on  how  much  knowledge  we 
accumulated  on  whether  or  not  we  could  write  down  information  that  could 
serve  as  a guide  for  designing  and  building  a tutoring  system.  The  experts 
were  assured  that  their  responses  would  remain  anonymous.  The  interviews 
lasted  an  average  of  1.5  hours. 
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6 EVALUATING  THE  SECOND  INTERVIEW 


Two  goals  existed  for  evaluating  the  results  of  the  second  interview. 
One  goal  was  to  confirm  or  refute  the  classification  scheme  devised  for  the 
selected  problem  solving  tasks  from  the  first  interview.  This  was 
important  in  order  to  determine  the  effectiveness  of  that  first  interview. 
The  second  goal  was  more  fundamental  with  respect  to  the  goals  of  the 
overall  project,  namely  to  determine  if  the  classification  scheme  and  the 
pairing  with  knowledge  acquisition  techniques  was  appropriate.  The  first 
goal  was  fairly  easy  to  assess  and  is  discussed  in  Sections  6.1  and  6.2. 
The  second  goal  is  much  harder,  and  is  addressed  in  Section  6.3. 

Once  a second  interview  for  a task  was  performed,  we  again  rated  the 
tasks  along  the  set  of  characteristics.  However,  this  set  consisted  of  124 
characteristics  because  we  broke  the  training  characteristic  used  in  the 
initial  ratings  into  general  education  vs.  specific  training  in  the 
operational  environment  for  each  of  the  sixteen  tasks.  This  allowed  the 
impact  that  the  existence  of  formal  training  seems  to  have  on  knowledge 
acquisition  to  be  explicitly  represented. 

The  rating  based  on  the  second  interview  was  performed  independently 
of  the  ratings  given  after  the  first  interview.  This  would  allowed  us  to 
look  at  how  much  the  ratings  changed  from  one  interview  to  the  next,  thus 
providing  some  indication  of  how  our  impression  of  each  task  changed. 
Based  on  the  ratings  given  for  the  124  characteristics  for  each  of  the 
sixteen  tasks,  the  same  analyses  were  performed  on  the  resulting  matrix  to 
see  if  any  significant  changes  occurred  in  how  the  tasks  clustered  based  on 
the  second  interview.  These  analyses  are  discussed  further  below. 


6.1  A Preliminary  Look  At  The  Data 


The  same  informal  data  analyses  that  were  performed  on  the  ratings 
from  the  first  set  of  interviews  was  performed  on  the  ratings  from  the 
second  set  of  interviews.  The  results  of  these  analyses  appear  in  Appendix 
L.  A comparison  between  the  analyses  from  interview  one  and  two  appear 
below. 


6.1.1  Pairwise  Comparison  - 

The  pairwise  comparison  for  the  second  interview  did  differ  some  from 
the  first  interview.  For  example,  pilot  training  did  not  dominate  quite  so 
heavily  the  position  of  being  least  similar  to  any  other  task.  It 
constituted  only  four  of  the  first  ten  entries  in  the  table,  whereas  in  the 
first  interview  it  constituted  nine  of  the  first  ten  entries.  However, 
pairs  containing  pilot  training,  air  traffic  control,  or  surgery  made  up 
the  first  ten  pairs  in  the  least  similar  listing.  In  the  set  of  the  ten 
least  similar  task  pairings,  one  of  these  three  appeared  with  such  tasks  as 
DRAIR  generation,  accounting,  language  training,  leadership,  and  program 
management.  These  pairings  illustrate  the  distinction  between  the  tasks 
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that  have  physically-based  component  and  some  of  the  tasks  that  are  larger 
in  scope  and  less  well-defined.  In  addition,  the  less  well-defined  tasks 
in  these  pairings,  along  with  form  fill-out,  protocol  design,  and  software 
design  again  constitute  the  majority  of  the  pairings  in  the  twenty-five 
pairs  that  are  considered  most  similar  (the  last  25  pairings  listing  in  the 
similar  listing).  Thus,  the  primarily  knowledge-based  tasks  still  cluster, 
with  the  addition  of  language  training,  as  they  did  in  the  analysis  from 
the  first  interview.  The  remaining  tasks,  cargo  loading,  weather 
prediction,  equipment  diagnosis,  and  medical  diagnosis,  appear  to  be 
scattered  among  the  most  and  least  similar  tasks  as  they  were  in  the 
initial  analysis.  Thus,  though  certain  differences  appear  between  the 
analyses  from  the  first  and  the  second  interview,  the  basic  relationships 
still  hold  between  the  sixteen  tasks  based  on  their  characteristic  ratings. 


6.1.2  Ordered  Characteristic  Count  - 

The  major  difference  between  the  characteristic  counts  from  the  first 
and  the  second  interviews  is  that  in  the  second  interview  we  rated  many 
more  characteristics  with  a zero.  This  was  probably  due  to  the  fact  that 
the  second  interview  helped  us  to  focus  on  the  specific  task  and  thus,  many 
characteristics  became  less  relevant.  Based  on  the  analysis  from  the 
second  interview,  the  following  characteristics  appear  to  be 
non-distinguishing  because  they  have  a 0 rating  for  most  of  the  tasks: 
inductive,  kinesthetic,  perceptual  speed,  gross  (motor  skills), 
contaminants,  lighting,  scan  display,  repetitive,  fatigue,  negotiates, 
induce,  response  chaining,  code,  introspective  accuracy,  schedule,  physical 
strain,  instructs,  confinement  electricity,  relfex,  analogical,  compound, 
complex  continuous,  interpretive  movement,  weight,  computerize,  test 
equipment,  analogs,  acceleration,  vehicles,  height,  gender,  weapon  systems, 
sleep,  magnetism,  and  isolation.  This  list  constitutes  14  more 
characteristics  where  thirteen  or  more  tasks  received  a rating  of  zero  for 
the  characteristic  than  did  in  the  rating  after  the  initial  interview.  In 
the  analysis  of  the  data  from  the  second  interview,  no  characteristics 
seemed  to  be  non-distinguishing  due  to  a high  count  in  four  ratings  (10  or 
more  tasks  getting  a four  rating  for  the  characteristic).  Many  of  the 
characteristics  that  had  high  counts  in  the  0 and  4 ratings,  and  lesser 
counts  in  the  middle  ratings  of  1,  2,  and  3 from  the  first  interview  did 
not  have  nearly  the  concave  shape  to  their  rating  numbers  based  on  the 
second  interview.  Thus,  these  characteristics  may  not  be  as  distiguishing 
as  they  might  have  appeared  to  be  initially. 


6.1.3  Important/Unimportant  Characteristics  By  Task  - 

As  with  the  data  from  the  first  interview,  this  informal  data  analysis 
examined  the  most  and  least  important  characteristics  for  each  task.  A 
list  was  generated  for  each  task  that  provided  the  characteristics  that 
received  a zero  rating  and  a four  rating  for  each  task.  This  provided  a 
means  of  seeing,  by  task,  which  were  key  attributes  and  which  were  not. 
These  lists  appear  in  Appendix  L.  In  addition,  the  0-rated  and  4-rated 
characteristics  are  listed  for  each  task  in  the  Technical  Journal,  Appendix 
B • 
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6.2  Statistical  Analyses 


The  second  ratings  of  the  characteristics  entered  into  a factor 
analysis  and  cluster  analysis.  The  characteristics  entered  into  the 
analyses  included  the  additional  characteristics  concerning  training  and, 
as  in  the  first  analyses,  we  collapsed  the  scores  across  a category  of 
characteristics  if  a summary  score  for  that  category  made  sense. 


6.2.1  Factor  Analysis  Of  The  124  Task.  Characteristics  - 

An  inspection  of  the  eigenvalues  of  the  principle  components  factor 
analysis  lead  to  three  factors  being  retained.  The  three  factors  were 
rotated  with  a varimax  rotation  to  allow  for  correlated  factors. 

Factor  1 accounted  for  30£  of  the  variance.  Characteristics  such  as 
cover  and  differentiate,  inputs  to  the  problem,  problem  complexity, 
perceptual  requirements,  psychological  stressors,  spatial  reasoning, 
temporal  reasoning,  and  communication  loaded  heavily  on  this  factor.  This 
factor  appears  to  represent  a cluster  of  characteristics  associated  with 
very  complex,  very  demanding  tasks.  Factor  2 accounted  for  15%  of  the 
variance  and  is  associated  with  characteristics  such  as  case-based 
reasoning,  generalization,  knowledge  rich  aspects,  and  no  perceptual, 
psychological  or  physical  requirements.  This  set  of  characteristics 
describes  tasks  that  are  cognitively-oriented,  requiring  a knowledge  of 
past  problems  but  with  no  physical  component.  Finally,  factor  3 accounted 
for  15%  of  the  variance  and  is  associated  with  the  use  of  means-ends 
analysis  in  problem  solving  and  data  oriented  heuristic  search.  This 
represents  a set  of  tasks  in  which  the  data  drives  a means-ends  search  of 
the  problem  space. 


6.2.2  Cluster  Analysis  Of  The  16  Tasks  - 

An  eight  cluster  solution  revealed  the  following  task  clusters: 

1.  software  design  and  leadership  training 

2.  air  traffic  control,  surgery,  and  pilot  training 

3.  cargo  loading,  accounting,  program  management,  form  fill-out,  and  drair 
generation 

4.  console  operations  and  medical  diagnosis 

5.  protocol  analysis 

6.  weather  forecasting 

7.  equipment  diagnosis 
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8.  foreign  language  training 

Software  design  and  leadership  training  clustered  because  of 
correspondence  on  the  following  characteristics:  temporal  reasoning, 
case-based  reasoning,  means-ends  analysis,  forming  sub-goals,  generate  and 
test  strategy,  and  problem  decomposition.  These  tasks  also  require 
communication  between  participants. 

Air  traffic  control,  surgery,  and  pilot  training  have  both  knowledge 
rich  and  high  performance  components.  They  are  different  than  most  of  the 
other  tasks  we  examined  because  they  require  so  much  input  to  the  task  and 
the  input  is  very  complex.  These  tasks  clustered  because  they  can  be 
characterized  as  tasks  requiring  spatial  reasoning,  means-ends  analysis, 
formation  of  sub-goals,  use  of  formula/procedure/algorithm,  problem 
decomposition,  a large  number  of  inputs  to  the  problem,  complexity,  and 
perceptual,  psychological,  and  physical  requirements. 

The  third  cluster  is  similar  to  the  third  cluster  obtained  in  the 
cluster  analysis  of  the  initial  ratings  but  it  lacks  leadership  training 
and  equipment  diagnosis.  This  roughly  corresponds  to  our  breakout  of  tasks 
that  require  a lot  of  background  knowledge  but  are  not  trained  formally  for 
the  environment  in  which  they  will  be  performed.  These  tasks  cluster 
because  of  correspondence  on  the  following  characteristics:  formation  of 
sub-goals,  use  of  formula/procedure/algorithm,  a moderate  amount  of  input 
to  the  problem  solving  process,  a lot  of  communication  processes,  no 
statistical  or  spatial  reasoning,  and  no  perceptual,  environmental, 
psychological,  or  physical  requirements. 

Console  operations  and  medical  diagnosis  are  both  diagnostic  tasks. 
They  require  the  problem  solver  to  engage  in  temporal  and  case-based 
reasoning  and  to  model  the  system  states.  Also,  they  are  data  driven, 
utilize  a cover  and  differentiate  problem  solving  strategy,  have  some  use 
of  procedures,  and  occasionally  require  the  problem  solver  to  return  to  the 
definition.  Finally,  there  are  some  perceptual,  environmental,  and 
psychological  requirements  for  task  performance. 


6.2.3  Summary  - 

The  factor  analysis  revealed  three  main  factors  in  the  questionnaire. 
The  first  factor  consisted  of  characteristics  associated  with  tasks 
involving  a physically-based  skill,  having  perceptual  requirements,  and 
requiring  spatial  and  temporal  reasoning.  The  second  factor  corresponded 
to  knowledge  rich  tasks  involving  no  perceptual  requirements  or 
environmental/psychological  stressors.  The  third  factor  was  related  to 
tasks  involving  means-ends  heuristic  search  in  a data-driven  fashion. 

As  in  the  first  set  of  analyses,  these  factors  were  related  to  the 
clustering  solution  in  the  cluster  analysis.  Air  traffic  control,  surgery, 
and  pilot  training  as  well  as  console  operations  and  medical  diagnosis  have 
physical  and  perceptual  requirements.  The  remaining  tasks  can  be 
characterized  as  primarily  knowledge-based  and  involving  no  perceptual 
requirements  or  environmental/psychological  stressors.  The  relationship  of 
the  third  factor  to  the  clustering  solution  is  less  clear.  Many  of  the 
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tasks  (e.g.,  medical  diagnosis,  console  operations,  and  weather 
forecasting)  are  data-driven  and  require  some  means-ends  analysis  (e.g., 
program  management  and  pilot  training). 

The  clusters  are  fairly  similiar  to  those  that  emerged  from  the  first 
ratings.  Cargo  loading,  accounting,  program  management,  form  fill-out,  and 
drair  generation  clustered  in  both  solutions  as  did  pilot  training  and 
surgery.  Weather  forecasting  and  foreign  language  training  never  clustered 
with  other  tasks.  In  the  second  clustering  solution,  medical  diagnosis  and 
console  operations  clustered  which  was  not  suprising  because  they  both 
involve  diagnosis.  There  was  inconsistency  between  the  cluster  from  the 
original  ratings  of  software  design  and  protocol  design  and  the  cluster 
from  the  second  ratings  of  software  design  and  leadership  training.  We 
determined  that  software  design  and  protocol  design  are  very  different 
tasks.  Protocol  design  involves  a knowledge  of  experimental  and 
statistical  methods  and  is  much  more  formalized  than  software  design. 
Software  design  is  a true  design  task,  involving  decomposition,  and 
propose/refine  and  generate/test  problem  solving  strategies. 

Also,  the  second  clustering  solution  confirmed  our  contention  that  two 
main  categories  of  tasks  exist:  those  that  are  formally  trained  and  those 
that  are  not.  Clusters  2,  4,  and  6 (containing  air  traffic  control, 
surgery,  pilot  training,  console  operations,  medical  diagnosis,  and  weather 
forecasting)  include  the  formally  trained  tasks.  With  the  exception  of 
cargo  loading,  the  others  are  not  formally  trained. 


6.3  An  Analysis  Based  On  Experience 


In  evaluating  the  results  of  the  second  interview  to  determine  how 
effective  the  knowledge  acquisition  approach  used  was,  a key  issue  arose 
concerning  how  one  can  determine  the  amount  of  knowledge  acquired  from  a 
knowledge  acquisition  session.  It  seemed  that  the  best  way  to  assess  the 
effectiveness  of  a knowledge  acquisition  technique  is  to  somehow  quantify 
the  amount  of  knowledge  that  was  obtained  in  a given  period  of  time.  In 
addition,  in  order  to  truly  determine  if  one  knowledge  acquisition 
technique  is  superior  to  another  in  a given  situation,  a formal  experiment 
must  be  run  that  is  designed  to  remove  all  potential  biases  from  the 
process,  such  as  variances  due  to  the  domain  experts  and  knowledge 
engineers.  Interviews  and  ratings  must  be  performed  independently.  Thus, 
to  fully  assess  the  effectiveness  of  the  second  interview  with  respect  to 
the  knowledge  acquisition  technique  was  well  beyond  the  resources  of  the 
current  effort.  Some  suggested  approaches  to  the  problem  are  discussed  in 
Section  7.2. 

In  a less  formal  way,  however,  we  did  have  one  data  point  concerning 
the  amount  of  knowledge  obtained  from  an  interview.  Though  most  of  the 
analysis  of  the  interviews  was  done  based  on  the  ratings  for  each  task,  we 
did  have  one  additional  source  of  data  for  each  interview  performed,  namely 
our  hand-taken  notes  from  the  interview  itself.  In  most  cases,  the  amount 
of  notes  taken  and  the  ease  with  which  they  were  obtained  seemed  to  provide 
a rough  assessment  of  the  success  of  an  interview  with  respect  to  the 
amount  of  knowledge  obtained.  For  example,  the  notes  from  one  interviewer 
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on  cargo  loading  were  10  pages,  on  air  traffic  control  seven  pages,  and  on 
weather  forecasting  10  pages.  We  felt  that  these  interviews  were  highly 
successful  due  to  the  amount  of  information  gained  and  the  organization  of 
that  information.  Most  of  the  other  interviews  were  only  two  to  four  pages 
in  length,  indicating  that  a lesser  amount  of  information  was  obtained. 
There  were  some  exceptions,  such  as  accounting,  surgery,  and  aircraft 
piloting,  where  fewer  notes  were  taken  because  some  much  of  the  information 
was  visual.  However,  in  general  fewer  notes  often  coincided  with  the 
feeling  that  less  information  was  obtained  because  it  was  not  available  or 
the  task  was  still  just  too  nebulous.  Striking  examples  of  this  latter 
situation  were  program  management  and  equipment  diagnosis. 


6.4  Results  And  Conclusions  From  The  Second  Interviev 


Based  on  the  results  of  the  second  interview,  the  delineation  between 
tasks  that  are  formally  taught  and  those  that  were  learned  more  or  less  on 
the  job  remained  from  the  first  to  the  second  interview.  Task  clusterings 
remained  very  similar,  though  the  characteristics  themselves  were  not 
always  rated  the  same  across  the  tasks.  When  examining  the  amount  of 
knowledge  we  felt  that  we  acquired  from  an  interview  and  the  ease  with 
which  it  was  acquired,  the  fact  that  a task  was  explicitly  trained  played  a 
tremendous  role.  Those  individuals  who  had  taught,  or  were  teaching,  the 
task  were  much  better  prepared  to  talk  about  the  task  in  an  organized  and 
explicit  fashion.  They  were  also  much  more  capable  of  generating  examples 
and  explaining  them  since  this  is  something  in  which  they  have  had 
considerable  practice.  When  the  task  is  not  formally  trained,  the 
knowledge  acquisition  process  is  much  more  dependent  on  the  skills  of  the 
knowledge  engineer  to  guide  the  interview  and  make  sense  of  the  information 
collected.  This  is  not  always  possible  in  only  a couple  of  hours  of 
discussion  with  an  expert.  However,  it  is  also  important  to  be  able  to 
detect  when  the  expert  does  not  know  enough  about  the  area  of  interest  and 
to  try  to  find  another  expert.  We  had  two  experts  in  this  effort  that,  if 
we  were  tasked  with  actually  building  a tutoring  system,  would  need  to  be 
supplemented  with  other  experts  more  involved  performing  of  the  actual  task 
on  a day-to-day  basis. 

Based  on  this  experience,  we  offer  the  following  rules  of  thumb  for 
developing  a second  interview  based  on  the  results  of  the  first  interview: 

o In  general,  if  the  task  has  a distinct  starting  and  stopping  point  and 
the  task  is  well-defined,  then  try  to  go  through  at  least  one  example 
in  the  second  interview.  This  example  should  be  selected  from  a 
continuum  of  easy  to  hard,  starting  with  easier  ones.  What  makes  the 
task  easier  or  harder  should  be  identified  up  front  and  explained  with 
each  example.  There  are  a number  of  exceptions: 

- If  the  problem  solving  task  extends  over  more  than  a couple  of 
hours,  then  you  should  break  the  task  down  further  into  sub-tasks 
and  examine  them  before  preceding  to  an  example. 
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- If  the  task  is  distinctly  procedural,  then  the  example  should  be 
used  only  as  a way  of  eliciting  the  general  procedure  and  following 
how  it  is  applied  to  a specific  situation.  There  is  no  point  in 
getting  specific  information  on  an  example  and  then  having  to  try 
to  generalize  from  it  when  the  generalization  exists  in  the 
expert's  head. 

- If  the  task  is  very  data  intensive,  then  a summary  of  the  relevant 
data  elements,  where  they  come  from,  and  how  they  are  used  should 
be  discussed  before  the  example  is  discussed. 

- If  the  task  uses  a complex  interface  such  as  a console,  cockpit,  or 
set  of  tools,  then  these  should  be  discussed  in  terms  of  their 
components  and  their  functions  before  going  into  an  example. 

- If  the  task  is  formally  taught,  then  the  examples  should  be 
selected  initially  from  the  early  part  of  the  curriculum  and 
discussed  as  they  progress  to  more  complex  ones. 


o If  the  task  is  ill-defined,  but  a curriculum  exists,  then  use  the 
curriculum  to  guide  the  interview  process  and  ignore  using  examples. 

o If  the  task  is  ill-defined  or  takes  too  long  to  go  through  an  example, 
then  the  interview  process  should  continue  to  explore  the  task  area  to 
determine  subtasks  that  can  be  examined  further. 

Thus,  the  recommendation  for  the  second  interview  is  heavily  example-based. 
However,  a word  of  caution  is  in  order.  The  examples  that  we  discussed 
with  the  experts  were  not  usually  easily  reduced  to  a series  of  attributes 
and  values  that  could  be  entered  into  a knowledge  acquisition  tool  for 
generalization  into  rules.  The  examples  often  had  many  aspects  and  were 
quite  complex.  In  addition,  though  the  examples  were  always  used  as  a way 
of  structuring  the  interview,  they  were  not  the  sole  source  of  knowledge 
from  the  interview.  The  experts  provided  many  insights  into  the  relevance 
of  various  attributes  and  their  effect  on  the  problem  solving  task  that 
would  be  lost  on  an  automated  knowledge  acquisition  tool.  In  addition, 
there  was  often  a general  procedure  implicitly  used  within  the  analysis  of 
a specific  example  that  would  not  necessarily  be  apparent  just  from  a few 
examples  or  even  from  many  hundreds  of  examples.  Thus,  just  a collection 
of  examples  from  which  we  could  generalize  would  be  a highly  inefficient 
and  ineffective  way  to  acquire  knowledge  for  building  a tutoring  system  or 
a knowledge-based  system.  Use  of  examples  can  provide  a much  richer  medium 
for  obtaining  knowledge  than  simply  the  collection  of  attributes  and  values 
as  they  relate  to  a solution.  Thus,  most  automated  knowledge  acquisition 
tools  available  today  are  at  an  extreme  disadvantage  because  they  do  not 
have  the  variety  of  methods  for  obtaining  knowledge  that  are  available  to  a 
human . 
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7 CONCLUSIONS 


The  results  of  this  research  are  necessarily  general.  Being  only  a 
small  initial  attempt  at  a fundamental  problem  in  artificial  intelligence 
and  cognitive  psychology,  the  process  has  had  to  be  bootstrapped  and  the 
results  can  only  provide  pointers  to  further,  more  formal  research  designs. 
Much  of  the  results  of  the  research  have  already  been  discussed  in  earlier 
sections  of  the  report.  Thus,  what  is  discussed  in  this  final  section 
involves  the  resulting  proposed  taxonomy,  how  we  would  approach  this 
problem  if  we  were  to  do  it  again,  and  recommendations  for  formal 
experimentation  that  could  lead  to  verification,  modification,  and 
enhancements  of  the  proposed  taxonomy. 


7.1  The  Proposed  Problem  Solving  Taxonomy  For  Knowledge  Acquisition 


The  proposed  problem  solving  taxonomy  to  support  knowledge  acquisition 
based  on  the  research  described  in  this  document  appears  in  Figure  2.  How 
this  taxonomy  evolved  has  been  discussed  in  previous  sections.  We  believe 
that  the  first  few  levels  of  the  taxonomy  including  how  the  task  was 
learned  by  the  expert,  the  complexity/data  intensiveness  of  the  task,  and 
the  continuity  with  which  the  task  is  performed  are  areas  that  have  not 
been  addressed  previously  by  a problem  solving  taxonomy  of  this  sort. 
Thus,  they  are  unique  findings  of  this  research.  Other  attributes,  such  as 
procedural  orientation,  issues  in  cognitive  vs.  high  performance,  and 
issues  is  construction  vs.  classification  have  been  discussed  by  other 
researchers.  However,  we  believe  that  these  latter  attributes  really  only 
affect  the  knowledge  acquisition  process  in  the  later  stages  of  a 
development  effort  and  to  a lesser  degree  the  first  three  levels. 

This  research  has  provided  an  efficient  and  effective  method  of 
approaching  the  first  few  interviews  in  the  knowledge  acquisition  process. 
Total  time  required  by  the  expert  is  between  two  and  four  hours.  By  using 
the  revised  first  interview  questionnaire  provided  in  Appendix  E and  rating 
the  task  characteristics  on  a scale  of  0-4  for  the  final  task 
characteristics  listed  in  Appendix  0,  a knowledge  engineer  should  be  able 
to  classify  the  task  according  to  the  taxonomy  in  Figure  2.  In  addition, 
the  rules-of-thumb  for  approaching  the  second  interview  that  are  provided 
in  Section  6.4,  along  with  the  classification,  should  provide  a means  for 
continuing  the  knowledge  acquisition  process  for  a given  problem  solving 
domain.  Though  many  of  the  characteristics  of  a task  that  are  evaluated  by 
this  approach  have  little  or  no  impact  on  the  knowledge  acquisition 
approach  to  take,  they  do  provide  input  to  other  design  decisions  in  the 
development  of  an  intelligent  tutoring  system  or  a knowledge-based  system. 
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7.2  Assessment  Of  Approach 


If  similar  informal  research  were  to  be  performed  again,  such  as  to 
verify  the  findings  or  duplicate  the  general  results,  much  of  the  approach 
that  we  used  we  believe  should  remain  unchanged.  (More  formal  research  to 
address  some  of  the  other  problems  encountered  in  the  effort  are  discussed 
in  Section  7.2.)  Using  two  interviewers  worked  well  to  help  maintain  a 
balance  to  the  interviews  and  the  assessment  of  those  interviews.  The  data 
analysis  techniques  used  proved  to  be  relevant  and  useful.  However,  based 
on  our  experience,  both  the  task  characteristics  list  and  the  initial 
interview  could  be  refined  and  improved.  These  two  issues  are  discussed 
below. 


7.2.1  Changes  In  The  Task  Characteristics  List  - 

The  list  of  characteristics  actually  changed  very  little  over  the 
course  of  the  study.  Early  on  we  added  general  health  and  verbosity  to  the 
list  of  personal  characteristics.  Also,  we  added  boredom  to  the  list  of 
psychological  stressors  and  negotiates  to  the  list  of  communication 
processes.  However,  we  added  these  characteristics  early  enough  to  rate 
them  following  the  first  round  of  interviews. 

During  the  first  set  of  ratings,  we  noticed  that  certain 
characteristics  were  not  meaningful  to  us.  For  example,  we  were  uncertain 
how  to  rate  characteristics  such  as  difficulty,  analysis,  expertise,  and 
perceived  probability  of  success.  Additionally,  we  found  that  some  of  the 
characteristics  were  redundant;  inductive  reasoning  was  listed  under 
reasoning  techniques  and  under  problem  solving  tasks.  However,  at  this 
point  we  elected  to  retain  the  redundancies  through  the  second  interview. 

When  planning  the  second  round  of  interviews,  we  realized  that  our 
characteristic  "education/training"  was  very  influential  in  the  way  tasks 
are  taught  and  in  how  to  acquire  knowledge  from  an  expert  in  the  task. 
However,  we  originally  rated  this  as  one  characteristic;  it  is,  in  fact, 
two.  On  the  second  round  of  interviews,  we  rated  education  and  training 
separately. 

During  the  planning  process,  we  also  recognized  that  the 
characteristics  in  the  list  fell  into  at  least  three  categories:  those 
relevant  for  knowledge  acquisition,  those  relevant  for  system  design,  and 
those  that  were  riot  particularly  relevant  to  developing  knowledge-based 
systems.  Some  of  the  characteristics  dictated  how  we  chose  to  approach  the 
second  interview.  For  example,  the  amount  of  formal  training  (not 
education)  for  a task  determined  whether  or  not  we  recommended  that  the 
second  interview  involve  looking  at  the  training  curriculum.  If  a training 
curriculum  exists  it  greatly  accelerates  the  knowledge  acquisition  process. 
Also,  if  a task  is  governed  by  a formula,  procedure,  or  algorithm,  it  is 
important  to  discover  that  process  early  in  the  knowledge  acquisition 
process. 

The  second  category  of  characteristics  is  more  relevant  for  how  a 
knowledge-based  or  intelligent  tutoring  system  should  be  designed  for  that 
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task.  For  example,  if  the  task  is  performed  in  a noxious  environment  or  is 
very  psychologically  stressful,  the  design  of  an  intelligent  tutoring 
system  for  the  task  should  include  a method  of  stressing  the  student  to 
simulate  performance  under  adverse  circumstances.  Additionally,  if  the 
perceptual  requirements  of  the  task  are  high,  a student  interface  should 
have  fidelity  to  the  actual  task  in  order  to  ensure  good  transfer  to  the 
operational  environment. 

The  final  category  of  characteristics  were  those  that  did  no  have  much 
relevance  to  knowledge  acquisition  or  system  design.  Characteristics  such 
as  non-discursive  communication,  physical  characteristics,  and  whether  or 
not  the  task  was  technical  did  not  influence  our  knowledge  acquisition 
strategy  nor  would  they  influence  a system  design. 

Appendix  0 contains  the  final  list  of  characteristics.  We  removed  the 
redundancies  and  those  characteristics  that  are  not  relevant  to  knowledge 
based  system  design.  This  final  list  consists  of  15  categories  of 
characteristics  and  119  total  characteristics.  We  also  combined  and 
re-labeled  categories  as  necessary  to  consolidate  the  list. 


7.2.2  Changes  In  The  First  Interview  - 

We  altered  the  initial  interview  after  completing  all  interviews  in 
the  study.  We  deleted  questions  that  were  unclear,  ambiguous,  vague, 
redundant,  or  useless.  For  example,  the  initial  interview  contained  a 
question  about  interaction  with  others  during  each  of  the  four  phases  of 
problem  solving  (i.e.,  understanding  the  problem,  devising  the  plan, 
carrying  out  the  problem,  and  looking  back).  We  had  originally  included 
the  four  identical  questions  to  ensure  that  we  would  receive  information 
about  interaction  across  all  of  the  phases.  However,  the  breakdown  of  the 
interview  into  the  four  phases  of  problem  solving  was  transparent  to  the 
expert  during  the  interview.  We  did  not  want  to  confuse  them  with  aspects 
of  our  approach  that  were  not  relevant  to  their  role  in  the  effort.  As  a 
result,  this  redundancy  was  unnecessary  because,  when  experts  responded  to 
a question,  they  discussed  interaction  throughout  the  problem  solving 
process.  In  addition,  we  added  several  questions  that  tried  to  get  at  the 
education  versus  training  issue  for  a task.  Further,  we  re-ordered  many  of 
the  questions  so  that  the  interview  would  flow  more  smoothly.  Appendix  E 
contains  the  revised  first  interview. 


7.3  Recommended  Future  Directions 


This  research  project  was  not  large  enough  to  provide  the  opportunity 
to  test  the  hypotheses  we  generated.  Over  the  course  of  the  project,  we 
generated  an  initial  knowledge  acquisition  interview,  a hypothesis  about 
the  task  characteristics  that  are  of  particular  importance  to  knowledge 
acquisition,  and  a set  of  hypotheses  about  how  to  proceed  with  knowledge 
acquisition  based  on  the  results  of  the  first  interview.  To  validate  our 
assertions,  we  would  need  to  perform  experimental  research  that  would 
confirm  or  deny  our  hypotheses.  These  studies  would  evaluate  the  initial 
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interview  and  the  relationship  of  the  sequential  and  componential 
categorization  to  knowledge  acquisition. 


One  of  the  first  problems  we  encounter  when  attempting  to  design  a 
follow-on  study  to  evaluate  the  initial  interview  is  the  issue  of 
determining  how  much  knowledge  is  gained  in  an  interview.  As  mentioned 
earlier,  there  is  no  clear  way  to  quantitatively  measure  knowledge 
acquisition.  Even  general  qualitative  measures  are  elusive  as  well.  What 
defines  an  element  of  knowledge  (like  an  element  of  information  in 
information  theory)?  It  may  not  even  make  sense  to  talk  in  those  terms 
since  knowledge,  by  definition  involves  the  structuring  of  elements  as  well 
as  the  elements  themselves.  If  the  knowledge  engineer  can  design  a system, 
is  that  the  sign  of  adequate  knowledge  acquisition?  Or  is  it  whether  or 
not  the  system  performs  exactly  like  the  expert.  Even  so,  the  ability  to 
design  and  implement  a system  depends  on  the  skill  of  the  knowledge 
engineer  and  may  not  necessarily  reflect  the  amount  of  knowledge  obtained 
during  the  knowledge  acquisition  process. 


Because  of  these  problems,  we  would  need  to  resort  to  subjective 
ratings  of  the  effectiveness  of  knowledge  acquisition.  An  unbiased 
observer  would  rate  the  knowledge  acquisition  interview  on  a series  of 
scales.  In  this  way,  we  could  test  our  initial  interview  against  a variety 
of  other  verbal  knowledge  acquisition  techniques.  There  would,  of  course, 
need  to  be  adequate  control  by  having  raters  rate  all  interviews  in 
counterbalanced  order  and  by  keeping  knowledge  engineers  blind  to  the 
hypotheses. 

To  test  our  hypotheses  about  the  relationship  of  task  characteristics 
to  knowledge  acquisition,  we  would  need  to  select  two  tasks  in  each 
category  and  perform  knowledge  acquisition  appropriately  with  one  and 
inappropriately  with  the  other.  Independent  raters  would  evaluate  the 
sessions  on  a series  of  scales.  This  study  would  have  to  be  tightly 
controlled  because  of  the  tendency  of  knowledge  engineers  (and  all  humans 
engaged  in  interaction)  to  try  to  repair  an  interaction  that  is  going 
badly.  In  addition,  tasks  would  have  to  be  paired  carefully  and  different 
knowledge  engineers  would  need  to  perform  the  good  and  bad  techniques.  (We 
would  covary  out  variance  due  to  the  task  type  and  knowledge  engineer  in 
the  statistical  analyses.) 

Figure  2 shows  a possible  knowledge  acquisition  taxonomy  that  has  been 
developed  based  on  the  research  reported  on  here.  One  hypothesis  that  this 
study  also  generated  was  that  one  of  the  most  important  distinctions  for 
knowledge  acqusition  is  whether  the  task  is  sequential  or  componential. 
This  is  the  first  layer  in  the  taxonomy.  The  componential/sequential 
distinction  proposed  in  this  study  is  a superordinate  level  and  the  lower 
levels  also  influence  the  knowledge  acquisition  process,  although  to  a 
lesser  extent.  Lower  levels  separate  complex  from  non-complex  tasks  and 
procedural  from  non-procedural  tasks.  Although  we  didn't  evaluate  the 
relationship  of  knowledge  acquisition  farther  than  the  fourth  level,  it  is 
likely  that  distinctions  discussed  by  other  researchers  influence  knowledge 
acquisition  also.  Studies  to  validate  this  hierarchy  are  important  for  the 
goal  of  formalizing  the  knowledge  acquisition  process. 
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LIST  OF  CHARACTERISTICS 


1.  Reasoning  Techniques  - characteristics  related  to  the  use  of  information  to  perform  the  task 

statistical 

spacial 

temporal 

analogical 

case-based 

modelling 

mathematical 

deductive 

inductive 

2.  Problem-Solving  Techniques  - search  strategies  for  finding  a solution 

means-ends  analysis 
forming  sub-goals 
generate-and-test 

heuristic  search  - goal-directed/data-driven 

cover  & differentiate 

propose  & refine 

acquire  & present 

formula/procedure  / algorithm 

decomposing 

recombining 

generalizing 

specialization 

return  to  definition 

3.  Inputs  - information  input  to  the  task 

visual 

verbal 

auditory 

kinesthetic 

written  material 

instrumentation/test  results 

man/machine  interface 

databases 

historical  information 
propadeutics 


Task  Complexity  - characteristics  of  the  data  array  and  problem  solving  space  that  influence 
the  task 

branching  factor 
dynamism 

time  factors/constraints 

uncertainty 

difficulty 

Technical  Dimension  - the  degree  to  which  the  task  requires  a technical  background  or  skill 
high  to  low 

quantitative  vs.  qualitative 

Motor  Processes  - physical  manipulation  during  the  task 

Complex-continuous 

compound 

reflex 

simple-discrete 

fine 

gross 

repetitive 

Information  Processing  - characteristics  of  the  way  data  is  manipulated 

categorize 

calculate 

code 

computerize 

interpolate 

itemize 

learn 

tabulate 

translate 

Problem  Solving  Tasks  - cognitive  operations  on  the  data 

analyze 

deduce 

induce 

choose 

compare 

compute 

estimate 

integrate 

plan 

supervise 

monitor 

interpret 


9. 


Recall  - the  role  of  memory  in  the  task 


facts 

principles 

procedures 

analogs 

cases 

10.  Perceptual  - use  of  the  visual  input  during  task  performance 
perceptual  speed 

searching  for  and  receiving  information 
identifying  objects,  actions,  and  events 
scanning  a display 

11.  Environment  - physical  and  psychological  characteristics  of  the  environment  in  which  the 
task  is  performed 

11.1  Physical 

acceleration 

confinement 

isolation 

contaminants 

electricity 

lighting 

magnetism 

noise 

11.2  Psychological 

fatigue 
mental  strain 
stress 

physical  strain 
precision 

attention/automaticity 
response  chaining 
attention  span 
amount  of  sleep 
work  schedule 

12.  Personal  Characteristics  - characteristics  required  to  perform  the  task 
12.1  Physical 

age 

gender 

height 

weight 


12.2  Psychological 


education /training 
expertise 

introspective  accuracy /willingness 
perceived  probability  of  success 

13.  Type  of  Domain 
knowledge-rich  to  high  performance 

14.  Hardware/Equipment/Tools  Involved  - items  or  devices  used  during  the  task 

computer 

vehicles/aircraft 

weapon  system 

instruments 

notation 

test  equipment 

15.  Communication  Processes  - communication  during  task  performance 

15.1  Verbal 

negotiates 

supervises 

transmits 

requests 

instructs 

informs 

indicates 

directs 

communicates 

answers 

advises 

15.2  Nonverbal 

interpretive  movement 
expressive  movement 
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NOTES  ON  THE  TASKS  USED  IN  THE  KNOWLEDGE  ACQUISITION  INTERVIEWS 


The  research  performed  in  this  project  has  centered  on  the 
development  of  a problem  solving  taxonomy  that  will  allow  a knowledge 
engineer  to  classify  a problem  solving  task  and  then,  based  on  the 
classification,  obtain  direction  on  how  to  proceed  with  the  knowledge 
acquisition  process.  The  effort  has  focused  on  how  the  first  and  second 
interviews  in  the  knowledge  acquisition  process  should  take  place.  The 
first  interview  has  been  designed  as  a general  set  of  questions  that  can 
be  used  to  identify  the  relevant  characteristics  of  the  problem  solving 

domain  to  be  studied.  It  should  last  one  to  two  hours,  depending  on  the 

subject  matter.  From  this  interview,  the  knowledge  engineer  should  be 
able  to  fill-in  values  between  zero  and  four  indicating  the  relevance  of 
each  of  123  problem  characteristics  to  the  problem  solving  task.  The 
approach  to  the  second  interview  is  then  recommended  based  on  the  way 
that  the  problem  solving  task  is  rated  in  certain  categories.  The  second 
interview  should  also  last  one  to  two  hours,  depending  on  the  subject 

matter.  Some  "Rules  of  Thumb"  are  emerging  to  help  guide  this  approach 

to  the  second  interview.  For  example,  the  preferred  approach  to  the 
second  interview  appears  to  be  to  examine  an  example,  no  matter  what  the 
type  of  task.  This  example  can  be  specific,  but  it  should  also  be 
generalizable  so  that  some  of  the  basic  rules-of-thumb  and  procedures  can 
become  apparent.  However,  many  exceptions  occur: 

1.  If  the  task  is  continuous  or  requires  skills  that  are  not  easily 
observed  visually  with  commentation,  then  watching  is  not  very  useful 
and  a piece/level/component  of  the  task  should  be  selected  for 
discussion.  This  is  best  selected  from  the  curriculum  for  teaching 
the  task  (ie.  foreign  language  training). 

2.  If  a task  cannot  be  observed  firsthand  due  to  restrictions  of  access, 
a verbal  walk-through  of  an  example  from  actual  experience  should  be 
used  (ie.  console  operations,  surgery). 

3-  If  a specific  example  will  not  be  generalizable  due  to  the 

complexity,  amount  of  background  knowledge  required,  etc.  then  use 
the  example  as  a structure  and  have  the  expert  generalize  to  the 
rules  that  govern  the  decisions  that  are  made  (ie.  form  fill-out). 

4.  If  the  task  consists  of  a number  of  subtasks,  then  select  a subtask 

for  further  discussion  before  going  into  examples  (ie.  software 

design) . 

5.  If  the  task  involves  observing/assembling/assessing  a large  amount  of 

data  in  a dynamic  situation,  then  watching  actual  examples  will  not 
be  of  value  because  the  knowledge  engineer  will  not  be  able  to 
discern  what  is  relevant  at  a given  point  in  time  from  what  is  not. 
Then  the  knowledge  acquisition  process  should  center  on  identifying 
the  data,  identifying  techniques  for  selecting  the  relevant  data,  and 

identifying  analysis  techniques  used  in  processing  the  data  to  make  a 

decision  (i.e.,  air  traffic  control). 
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6.  If  a procedure  exists  for  performing  the  task,  then  the  procedure 
should  be  followed  using  an  example.  This  will  allow  the  knowledge 
engineer  to  obtain  the  general  rules  while  seeing  how  they  apply, 
rather  than  having  to  extract  the  rules  strictly  from  examples  (i.e., 
equipment  diagnosis). 

7.  If  equipment  or  tools  are  used  in  the  task  (i.e.,  cockpit  in  aircraft 
piloting,  surgical  tools  in  surgery)  then  the  second  interview  should 
include  a discussion  of  these  items  and  a description  of  what  they 
are,  how  they  work,  and  how  they  are  used. 

It  should  be  noted  that  both  the  first  and  second  interviews  are  oriented 
towards  identifying  and  acquiring  the  problem  solving  strategies  used  by 
the  subject  matter  experts.  Acquisition  of  lots  of  detailed  domain 
knowledge  that  guides  specific  decisions  is  not  the  goal  of  these  initial 
interviews.  Instead,  the  goal  is  to  develop  a problem  solving  framework 
for  a given  problem  solving  area  from  which  subsequent  knowledge 
engineering  sessions  can  be  designed  that  will  get  at  the  domain  details. 

Based  on  what  we  are  learning  from  the  second  interview,  we  still 
believe  that  observing  the  problem  solving  task  is  a good  idea  in  the 
second  round.  However,  it  should  be  understood  up  front  that  the 

knowledge  engineer  will  not  get  everything  that  goes  on  in  that  second 
interview.  Especially  with  tasks  that  involve  assimilation  of  a lot  of 
data  or  the  use  of  complex  equipment/interfaces  the  knowledge  engineer 
will  become  confused.  However,  we  believe  that  it  is  valuable  to 
actually  see  the  process  early  on  so  that  the  knowledge  engineer  can 
become  aware  of  all  of  the  different  things  that  will  have  to  be 
discussed  in  a later  interview. 


TYPE  A TASKS 


1.  Foreign  Language  Training 

Brief  Task  Description  - During  the  first  interview,  we  discussed  the 
requirements  of  her  job,  including  developing  the  curriculum,  course 
modification,  achievement  testing,  and  course  review.  The  curriculum 
must  account  for  four  levels  of  proficiency:  grammer,  vocabulary, 

language  skills,  and  language  functions. 

Expert  Characteristics  - Instructors  in  the  area  of  foreign  language 
training  usually  have  at  least  a Bachelor's  degree,  often  in  a 
foreign  language  or  a foreign  studies  program.  Individuals  at  the 
Defense  Language  Institute  who  teach  English  as  a foreign  language  do 
not  necessarily  need  to  know  another  language.  However,  it  is 
helpful  to  know  one  foreign  language  which  provides  a broader 
experience  in  language. 

Source  of  Expertise: 

Peggy  Goitia 

Defense  Language  Institute 
English  Language  Center 
Lackland  AFB,  Texas 
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(512)  671-2991 

Characterization  of  the  Task  Based  on  Interview  1 - We  rated  the 
following  characteristics  as  essential  to  training  a foreign 
language:  forming  sub-goals,  acquire  and  present  material,  decompose 
the  problem,  recombine,  verbal  and  written  inputs,  instrumentation, 
dynamism,  translation,  planning,  supervising,  monitoring, 
interpreting,  recall  of  facts  and  principles,  introspective  accuracy, 
articulation,  answering  questions,  communication,  directing, 
indicating,  informing,  requesting,  transmitting,  and  supervising. 
According  to  a pairwise  comparison  of  the  raw  data,  training  a 
foreign  language  was  most  similar  to  leadership  training  (90)  and 
least  similar  to  surgery  (67).  Foreign  language  training  emerged  as 
an  independent  cluster  in  the  eight-cluster  analysis. 

Approach  - The  approach  to  knowledge  acquisition  should  be  based  on 
the  curriculum  and  how  the  skill  is  taught.  In  addition,  language 
training  can  be  broken  down  into  several  components,  including 
vocabulary,  grammar,  verb  conjugation,  noun/adjective  inflection, 
pronunciation,  etc.  Therefore,  we  should  look  at  two  to  four  weeks 
of  the  curriculum,  identify  the  levels  taught  of  each  of  the 
components,  and  query  concerning  how  each  of  these  are  taught  and 
why. 

Justifications  - Observing  the  process  of  actually  speaking  a 
language  will  not  be  very  enlightening  at  this  point.  Little  can  be 
gained  from  "watching"  the  task  because  it  is  so  continuous  and  much 
of  what  is  driving  decisions  is  not  visible.  In  addition,  when 
skilled  in  the  speaking  of  a language,  decisions  are  quite  often  not 
made  consciously.  Thus,  a level  of  skill  along  the  continuum  should 
be  selected  and  the  rationale  for  what  is  taught  and  how  should  be 
investigated. 

Results  - For  the  second  interview,  Peggy  had  assembled  all  of  the 
written  material  used  in  teaching  one  segment  of  English  as  a foreign 
language.  This  consisted  of  four  books,  two  for  the  instructor  and 
two  for  the  student.  We  examined  these  books  and  developed  an 
understanding  of  what  various  categories  of  information  are  taught 
and  how.  The  books  provided  a means  of  observing  how  the  four  major 
areas  they  use  in  teaching  language  (function,  grammar,  vocabulary, 
and  skills)  are  actually  used  in  teaching  English.  Because  Peggy 
also  had  other  level  books  available,  we  were  able  to  compare  the 
types  of  items  taught  in  the  four  areas  in  Lesson  1,  Book  1 and 
Lesson  1,  Book  24.  Earlier  lessons  are  very  dependent  on  visuals  and 
all  lessons  require  a laboratory  portion  where  the  student  spends 
time  listening  and  responding  to  audio  tapes.  This  approach  allowed 
us  to  get  into  quite  a bit  of  detail  while  still  discussing  the 
rationale  behind  the  curriculum. 

2.  Aircraft  Pilot 

Brief  Task  Description  - The  aircraft  pilot  maintains  control  of  the 
aircraft  from  take-off  to  landing.  The  task  requires  both  cognitive 
and  physical  skills.  The  pilot  must  observe  instrumentation 
indicating  altitute,  attitude,  speed,  direction,  etc.  and  adjust  the 
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accordingly.  Piloting  an  aircraft  can  be  broken  down  into 
sets  of  maneuvers  such  as  take-off,  banking  at  various  angles  and 
speeds,  climbing,  and  landing.  In  military  piloting,  pilots  are 
briefed  before  flight  on  the  mission  and  debriefed  after  the  mission 
concerning  what  actually  happened. 

Expert  Characteristics  - An  aircraft  pilot  must  be  in  general  good 
health.  Training  is  very  regimented  and  involves  an  apprenticeship 
approach  where  the  student  pilot  goes  up  with  an  instructor  pilot  for 
a minimum  number  of  flying  hours.  The  student  is  initially  taught 
key  maneuevers  and  then  is  expected  to  put  them  together  into  a 
complete  mission.  Training  is  very  dependent  on  the  class  of 
aircraft  and  somewhat  dependent  on  the  specific  aircraft  type. 
Instructors  receive  specific  training  in  order  to  become  a pilot 
instructor. 

Source  of  Expertise: 

Bill  Ercoline,  Pilot  Instructor 
Building  170 
Brooks  Air  Force  Base 
(512)  536-3521 

Characterization  of  the  Task  Based  on  Interview  1 - Aircraft  pilot 
received  a rating  of  four  in  the  following  attributes:  spacial, 
temporal,  deductive,  sub-goals,  acquire-and-present , procedural, 
visual,  kinesthetic,  instrumentation,  man-machine-interface, 
branching,  dynamism,  constraints,  uncertainty,  simple-discrete, 
compare,  monitor,  procedures,  perceptual-speed,  scan-display, 
confinement,  noise,  cognitive-attentiveness,  general-health, 
articulation,  vehicles,  instruments,  advises,  answers,  directs, 
indicates,  informs,  and  instructs.  The  task  received  a rating  of 
zero  in  the  following  attributes:  statistical,  mathematical, 

generateandtest,  coveranddif ferentiate,  proposeandref ine,  recombine, 
databases,  historical,  propadeutics,  compound,  reflex,  calculate, 
code,  computerize,  interpolate,  learn,  tabulate,  translate,  compute, 
estimate,  isolation,  contaminants,  electricity,  lighting,  magnetism, 
sleep,  boredom,  computer,  weapon  systems,  test  equipment,  negotiates, 
and  expressivemovement.  According  to  a pairwise  comparison,  aircraft 
pilot  is  most  similar  to  to  air  traffic  control  (82)  and  least 
similar  to  DRAIR  generation  (51).  It  clustered  with  surgery  in  the 
eight-cluster  analysis. 

Approach  - The  approach  should  be  based  on  the  curriculum  and  how  the 
skill  is  taught.  Therefore,  we  should  look  at  a set  of  two  to  four 
maneuvers  and  go  through  how  each  is  trained.  These  maneuvers  need 
to  be  broken  down  into  how  the  student/pilot  manages  the  various 
attributes  of  the  maneuver  such  as  speed,  direction,  altitude,  etc. 

Justifications  - Observing  the  process  of  flying  an  aircraft  will  not 
be  very  enlightening  at  this  point.  Little  can  be  gained  from 
"watching"  the  task  be  performed  because  it  is  so  continuous  and  much 
of  what  drives  decisions  is  not  visible.  Correlation  of  pilot 
behaviors  to  the  response  of  the  aircraft  would  be  difficult  without 
further  knowledge  of  how  the  behaviors  effect  the  aircraft.  In 
addition,  pilot  reaction  to  aircraft  behavior  would  be  difficult  to 
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understand.  Thus,  a level  of  skill  along  the  continuum  should  be 
selected  and  the  rationale  for  what  is  taught  and  how  should  be 
investigated . 

Results  - The  interview  with  Bill  Ercoline  took  place  in  an  aircraft 
training  mock-up.  This  allowed  us  to  observe  the  actual  displays  and 
devices  available  to  a pilot  during  flight.  We  began  the  interview 
with  the  intention  of  going  through  a number  of  maneuvers  taught 
early  in  the  curriculum  to  a new  pilot.  However,  we  found  that  it 
was  difficult  to  follow  the  expert  due  to  the  number  of  different 
items  used  in  the  cockpit  that  we  did  not  yet  understand.  In  the 
end,  we  began  to  ask  about  how  the  cockpit  is  set-up  and  the  types  of 
attributes  that  the  pilot  controls  in  an  aircraft  and  how.  This  made 
the  examples  much  more  understandable.  Thus,  eventhough  this  task 
did  not  appear  to  be  terribly  data  intensive,  a considerable  amount 
of  information  is  observed  and  processed  by  the  pilot  and  it  should 
be  discussed  in  general  before  going  into  specific  details  on 
particular  tasks. 

3.  Weather  Forecasting 

Brief  Task  Description  - Generating  a weather  forecast  involves  the 
collection  of  a large  amount  of  data,  analyzing  it  in  terms  of  its 
meaning  with  respect  to  effect  on  temperature,  wind  direction  and 
speed,  precipitation,  etc.  Data  is  usually  collected  early  in  the 
day  and  entered  into  one  or  more  models  run  by  the  National  Weather 
Service.  Based  on  the  output  of  the  models,  the  raw  data,  the 
weather  forecasters  expertise,  and  the  area  of  the  world  of  interest, 
a prediction  is  made,  in  written  format,  for  one  to  five  days. 

Expert  Characteristics  - Weather  forecasters  obtain  a lot  of  training 
in  the  specifics  of  data  collection  and  utilization  of  models.  They 
are  trained  in  meteorology  in  general.  However,  forecasting  is 
highly  dependent  on  the  area  of  the  world  in  which  the  predictions 
are  being  made  and  on  the  time  of  year.  Thus,  once  a forecaster  has 
received  general  training,  he/she  still  requires  some  experience  in 
the  area  of  the  world  in  which  he/she  will  perform  the  task.  No 
specific  physical  attributes  are  necessary  for  the  job. 

Source  of  Expertise: 

Mr.  James  Skowronski, 

Air  Force  Weather  Forecast  Instructor 
Chinoot  Air  Force  Base,  Illinois 
(217)  495-2645 

Characterization  of  the  Task  Based  on  Interviev  1 - Weather 
forecasting  received  a rating  of  four  in  the  following  attributes: 
spacial,  temporal,  model-based,  acquire-and-present,  visual, 
branching,  dynamism,  uncertainty,  interpolate,  learn,  analyze, 
induce,  estimate,  integrate,  facts,  principles,  procedures,  and 
notation.  The  task  received  a rating  of  zero  in  the  following 
attributes:  analogical,  case-based,  deductive,  means-ends, 

generate-and-test , cover-and-dif ferentiate,  proposeandref ine, 
decompose,  specialize,  return-to-def inition,  auditory,  kinesthetic, 
man-machine-interface,  databases,  complex-continuous,  fine,  gross, 
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repetitive,  code,  computerize,  itemize,  tabulate,  deduce,  plan, 
supervise,  analogs,  perceptual  speed,  scan-display,  acceleration, 
confinement,  isolation,  contaminants,  electricity,  lighting, 
magnetism,  noise,  fatigue,  mental-strain,  stress,  physical-strain, 
preciseness,  cognitive-attentiveness,  response-chaining,' 

attention-span,  sleep,  schedule,  boredom,  general  health,  age, 
gender,  height,  weight,  introspective-accuracy,  articulation' 
vehicles,  weapon-systems,  test  equipment,  advises,  answers, 
communicates,  directs,  indicates,  informs,  instructs,  requests! 
transmits,  supervises,  negotiates,  expressivemovement,  and 
interpretive-movement.  According  to  a pairwise  comparison,  weather 
forecasting  is  most  similar  to  to  DRAIR  generation  (94)  and  least 
similar  to  aircraft  pilot  (54).  It  remained  an  independent  cluster 
in  the  eight-cluster  analysis. 

Approach  - Ue  have  two  choices  - follow  the  forecaster  around  while 
he/she  is  generating  a forecast  or  select  an  area  of  the  curriculum 
and  go  into  some  detail.  Either  way,  we  need  to  focus  on  the  data 
collection  and  how  the  patterns  are  recognized  for  classifcation  into 
a particular  type  of  weather  forecast. 

Justifications  - Currently  we  are  leaning  towards  looking  at  a live 
example  with  expert  commentation  due  to  the  amount  of  data  that  is 
involved.  The  task,  though  seemingly  dynamic  and  continuous  to  some 
degree,  involves  basically  a snapshop  of  the  atmospheric  data  from 
which  interpretation  is  made.  Thus,  though  it  may  at  first  appear 
watching  the  task  would  not  be  possible,  since  there  is  a 
distinct  start  and  finish  to  the  task  and  it  is  apparently  fairly 
procedural,  watching  the  task  should  result  in  learning  something 
useful. 

Results  - We  asked  Jim  Skowronsky  to  produce  the  five-day  weather 
forecast  based  on  the  data  available  to  him  the  morning  that  we  were 
there  for  the  Champaign-Urbana  area.  We  began  by  examining  a set  of 
20  weather  charts,  taken  at  12  hour  intervals  for  four  different 
weather  phenomena:  vorticity,  humidity,  thickness,  and 

precipitation.  These  charts  are  generated  from  models  based  on  data 
points  scattered  throughout  the  area.  The  thickness  and  vorticity 
charts  were  used  to  identify  the  locaiton  of  warm  and  cold  fronts  and 
the  Jet  Stream.  These  were  drawn  on  copies  of  the  charts  using 
colored  pens.  This  was  a very  visual  task  and  he  used  a lot  of 
knowledge  of  weather  models  to  complete  the  task.  Once  these  charts 
were  analyzed,  a satellite  picture  was  obtained  that  shows  what  is 
actually  there.  This  satellite  picture  is  then  compared  with  the 
vorticity  and  thickness  charts  to  verify  differences  between  the 
actual  and  the  predicted  based  on  the  models.  The  last  step  is  to 
try  to  identify  items  that  will  effect  the  forecast,  such  when  any 
fronts  may  come  through  the  area  and  how  much  moisture  is  in  the  air. 
There  are  various  other  sources  of  data  available  to  the  forecaster 
now,  as  well,  that  are  on  computers.  These  computer  sources  allow 
the  forecaster  to  compare  various  up-to-the-minute  charts  next  to 
each  other  on  the  computer  screen.  In  addition,  the  forecaster  will 
check  the  previous  forecast  and  look  out  the  window  to  see  what  is 
going  on.  Then  a forecast  is  generated  that  contains  cloud  cover, 
temperature,  and  chance  of  precipitation  for  today,  tonight,  and  the 
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next  three  days. 

4.  Air  Traffic  Control 

Brief  Task  Description  - Air  traffic  control  involves  the  monitoring 
of  radar  screens  and  weather  data  in  order  to  ensure  that  aircraft 
can  travel  safely  through  a given  area.  An  air  traffic  controller  is 
responsible  for  the  aircraft  in  a particular  air  space  and  at  a 

particular  phase  in  their  flight  take-off,  landing,  flight, 

ground,  etc.  The  task  involves  the  observation  of  large  amounts  of 
data  that  change  dynamically.  Based  on  the  data  the  air  traffic 

controller  must  generate  a plan  for  each  aircraft  in  his/her  airspace 
and  communicate  with  the  pilots  of  those  aircraft  based  on  what  they 
must  do  in  terms  of  changing  altitute,  speed,  and  direction.  Time  is 

a critical  issue  an  air  traffic  controller  works  for  a set  number 

of  hours  and  shift  work  is  standard.  Difficult  of  the  task  depends 
on  the  level  of  activity  (number  of  aircraft)  that  must  be  handled  at 
one  time. 

Expert  Characteristics  - Air  traffic  controllers  undergo  a large 

amount  of  training  in  the  area  of  air  traffic  control,  learning  how 
to  read  radar  scopes,  communicate  with  pilots,  etc.  However,  much 
experience  is  required  at  the  actual  airport  where  the  controller  is 
to  work  in  order  for  the  controller  to  become  good.  Individuals  who 
are  air  traffic  controllers  tend  to  be  very  decisive  people.  They 
also  have  health  problems  due  to  the  stress  and  shift  schedules  and 
divorces  are  common. 

Source  of  Expertise: 

Mr.  Jim  Johnson,  Radar  Air  Traffic  Control  Instructor 

FAA  Training  Facility 

Oklahoma  City,  Oklahoma 

(405)  680-6932 

Characterization  of  the  Task  Based  on  Interview  1 - Air  traffic 

control  received  a rating  of  four  in  the  following  attributes: 
spacial,  temporal,  means-ends,  sub-goals,  decompose,  visual,  verbal,, 
instrumentation,  dynamism,  constraints,  choose,  estimate,  plan, 
supervise,  monitor,  facts,  procedures,  perceputal-speed, 
search-receive-information,  identify-object/action/event , 

scan-display,  fatigue,  mental-strain,  stress, 

cognitive-attentiveness,  attention-span,  sleep,  schedule, 
general-health,  instruments,  notation,  communicates,  directs, 
informs,  requests,  and  supervises.  The  task  received  a rating  of 
zero  in  the  following  attributes:  statistical,  analogical, 

mathematical,  deductive,  inductive,  generate-and-test , 

proposeandref ine,  formula-procedure-algorithm,  recombine,  generalize, 
specialize,  return-to-def inition,  auditory,  kinesthetic,  databases, 
historical,  branching,  complex-continuous,  compound,  gross, 
calculate,  code,  computerize,  tabulate,  translate,  analyze,  deduce, 
induce,  compute,  integrate,  analogs,  acceleration,  isolation, 
contaminants,  electricity,  lighting,  magnetism,  physical-strain, 
preciseness,  boredom,  age,  gender,  height,  weight, 
introspective-accuracy,  computer,  vehicles,  weapon-systems,  test 
equipment,  answers,  indicates,  negotiates,  express ivemovement,  and 
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interpretive-movement.  According  to  a pairwise  comparison,  air 
traffic  control  is  most  similar  to  to  surgery  (83)  and  least  similar 
to  form-fill-out  (58).  It  remained  an  independent  cluster  in  the 
eight-cluster  analysis. 

Approach  - The  key  should  be  to  determine  what  factors  influence 
priority  setting  and  how  priorities  are  set.  We  cannot  yet  watch  the 
task  actually  being  performed  because  there  are  too  many  things  going 
on  to  be  able  to  interpret  the  actions  of  the  expert,  and  they  can 
not  usually  discuss  what  they  are  doing  during  the  task  due  to  the 
dynamism  and  time  constraints.  Thus,  we  will  have  to  perform  an 
interview,  though  it  will  still  be  based  on  examples.  The  examples, 
however,  should  be  contrived  along  a level  of  complexity  so  that  the 
factors  that  affect  air  traffic  control  decision  making  can  be 
observed.  ' 

Justifications  - We  are  still  at  a higher  level  on  this  task  than 
some  of  the  others  because  we  cannot  not  yet  perform  any  kind  of  task 
breakdown.  This  may  be  due  to  the  dynamism  and  lack  of  distinct 
start  and  completion  points  in  the  task,  beyond  the  shift  time 
itself.  We  will  acquire  the  factors  and  priorities  by  looking  first 

at  the  simplest  example  problem  getting  one  aircraft  down.  Then 

we  will  increase  the  complexity  of  the  task  (ie.  add  another 
aircraft  into  the  scenario)  and  query  concerning  how  the  problem  is 
approached  and  why. 

Results  - The  interview  began  with  a discussion  of  the  simplest  case, 
one  aircraft  landing  at  an  airport  where  one  air  traffic  controller 
owned  all  of  the  airspace.  Discussion  centered  on  drawings  of  the 
RADAR  scope  on  the  blackboard  with  indications  of  airspace,  airport, 
and  aircraft.  We  then  moved  to  a more  complicated  scenario  and  it 
became  clear  that  higher  level  traffic  is  handled  by  partitioning  the 
airspace  around  an  airport  and  the  various  goals  of  the  aircraft 
(i.e.,  landing,  departing,  crossing  through).  The  task  builds  in 
complexity  based  on  the  number  of  aircraft  having  to  be  serviced  in  a 
given  period  of  time.  Since  this  changes  with  the  time  of  day,  an 
air  traffic  controller's  job  can  change  many  times  within  a single 
shift.  Though  we  were  discussing  examples,  they  were  not  detailed 
examples  so  the  general  rules  of  how  the  air  traffic  controller  could 
handle  the  situation  was  available.  This  approach  allowed  us  to 
acquire  a lot  of  information  about  how  decisions  are  made  and  how  the 
task  is  broken  down  into  subcomponents. 

5.  Console  Operations 

Brief  Task  Description  - The  console  operation  task  selected  was  the 
propulsion  console  in  Mission  Control  at  Johnson  Space  Center.  The 
console  is  used  by  flight  controllers  to  monitor  the  propulsion 
system  of  the  Space  Shuttle  during  a mission.  The  task  involves 
observation  and  interpretation  of  a large  amount  of  raw  data 
concerning  the  status  of  the  propulsion  system  during  the  various 

phases  of  a mission  lift-off,  orbit  (coast  vs.  burn),  and 

re-entry.  To  monitor  this  data,  the  flight  controller  must  be  adept 
at  manipulating  a variety  of  keyboards,  thumbwheels,  and  level 
switches  in  order  to  bring  up  data  on  VDT  screens  and  to  format  sets 
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of  indicator  lights.  The  task  can  be  very  time  critical  if  a problem 
arises,  but  is  often  very  quiet. 

Expert  Characteristics  - Flight  controllers  for  the  propulsion  system 
are  educated  in  areas  such  as  mechanical  engineering  and  fluid 
dynamics.  They  have  a broad  background  of  engineering-type  knowledge 
that  allows  them  to  understand  the  system  they  are  monitoring.  In 
addition  to  the  broad  background  knowledge,  they  receive 
apprenticeship-type  instruction  in  the  flight  controller  job  (the 
cognitive  part  of  the  task).  They  receive  little  to  no  training  in 
the  actual  manipulation  of  the  console.  This  they  gain  from  practice 
during  simulations.  General  health  is  important,  but  no  other 
physical  characteristics  are  relevant  to  proper  performance  of  the 
task. 


Source  of  Expertise: 

Mr.  Matt  Barry,  Propulsion  Flight  Controller 

NASA/Johnson  Space  Center 

Houston,  Texas 

(713)  483-0876 

Characterization  of  the  Task  Based  on  Interview  1 - Console 
operations  received  a rating  of  four  in  the  following  attributes: 
temporal,  deductive,  formula-procedure-algorithm,  visual,  verbal, 
instrumentation,  man-machine-interface,  dynamism,  constraints, 
simple-discrete,  fine,  repetitive,  categorize,  interpolate,  learn, 
analyze,  deduce,  plan,  monitor,  facts,  procedures,  perceputal-speed, 
search-receive-information,  identify-object/act ion/event , 

scan-display,  noixe,  cognitive-attentiveness,  general-health, 
computer,  communicates,  and  informs.  The  task  received  a rating  of 
zero  in  the  following  attributes:  statistical,  spatial,  analogical, 

means-ends,  generate-and-test , propose-and-refine,  recombine, 
generalize,  return-to-definition,  auditory,  kinesthetic,  databases, 
branching,  complex-continuous,  compound,  gross,  code,  computerize, 
itemize,  tabulate,  induce,  analogs,  acceleration,  isolation, 
contaminants,  electricity,  magnetism,  sleep,  gender,  height, 
introspective-accuracy,  vehicles,  weapon-systems,  instructs, 
supervises,  negotiates,  and  interpretive-movement.  According  to  a 
pairwise  comparison,  console  operations  is  most  similar  to  to 
aircraft  pilot  (78)  and  least  similar  to  software  design  (61).  It 
remained  an  independent  cluster  in  the  eight-cluster  analysis. 

Approach  - It  would  be  difficult  to  observe  the  task  actually  being 
performed  because  of  the  limited  access  to  MCC  during  a shuttle 
flight  or  simulation.  However,  we  could  have  the  expert  verbalize  a 
particular  task,  such  as  monitoring  during  liftoff  or  the  leak  detect 
procedure  for  the  propellant  tank. 

Justifications  - This  task  involves  a lot  of  data  acquisition  and 
monitoring.  This  sort  of  activity  is  difficult  to  observe  and  obtain 
relevant  knowledge  conerning  what  the  expert  is  looking  at  and  why. 
Thus,  a verbal  description  of  the  process  while  going  through  an 
example  that  is  not  actually  happening  at  the  time  would  be  the  most 
effective  way  to  get  at  what  the  expert  is  doing  and  why. 
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Results  - We  asked  Matt  Barry  to  discuss  the  problem  of  engine 
failures  and  how  they  are  manifested  in  the  console.  He  drew  a 
diagram  of  the  fuel  system  on  the  blackboard  in  order  to  illustrate 
how  the  system  works  and  the  components  he  was  talking  about.  Engine 
failures  can  be  divided  into  two  types,  engine  and  propellant.  The 
key  issue  in  performing  the  task  is  the  ability  to  see  the  changes  in 
the  pressure  readings  instantly  so  that  appropriate  actions  can  be 
taken  to  shut  the  engine  down.  In  order  to  be  able  to  perform  the 
task  rapidly,  the  flight  controller  memorizes  the  various  pressure 
and  temperature  readings  for  each  type  of  failure.  Where  the  problem 
gets  hard  is  when  there  is  more  than  one  failure  at  a time  because 
they  are  not  able  to  memorize  what  dual  failures  look  like,  so 
additional  reasoning  is  required  which  takes  time. 

6 . Surgery 

Brief  Task  Description  - When  patients  are  refered  to  a surgeon,  the 
surgeon  collects  a complete  history  and  performs  a complete  physical 
examination  to  confirm  the  diagnosis.  The  surgeon  attempts  to  rule 
out  alternative  diagnoses  and  analyzes  the  patient's  fitness  for 
surgery.  During  the  surgery,  the  surgeon  heads  a team  of  health  care 
professionals,  including  an  anesthesiologist,  2 or  3 assisting 
nurses,  and  1 to  4 assistants  who  are  possibly  residents  or  interns. 
The  surgeon  makes  decisions  that  range  from  how  to  incise  the  patient 
to  whether  or  not  tissue  should  be  removed.  The  surgeon  must  contend 
with  the  fact  that  each  patient's  anatomy  is  slightly  different  as 
well  as  with  the  numerous  complications  that  can  arise  during  the 
procedure.  In  addition,  the  surgeon  attempts  to  perform  the 
procedure  as  quickly  as  possible. 

Expert  Characteristics  - Surgeons  undergo  the  same  initial  education 

and  training  as  general  medical  doctors  They  undergo  a four  year 

medical  program  at  a medical  school.  Upon  completion  of  this  study, 
however,  they  then  specialize  and  undergo  additional  residencies  to 
obtain  the  necessary  training.  Surgery  can  then  specialize  further 
into  various  types  of  surgery  based  on  the  location  and  system  of  the 
body. 


Source  of  Expertise: 

Dr.  C.  William  Hall 
Cardiologist 

Southwest  Research  Institute 
(512)  522-2670 

Dr.  C.  William  Hall  completed  his  medical  degree  in  1956.  The  then 
completed  a 5 year  residency  in  thoracic  surgery.  He  was  part  of  a 
fellowship  program  at  Baylor  University,  during  which  he  was  part  of 
the  team  that  designed  and  implanted  the  first  blood  pump  in  humans. 
He  taught  at  the  medical  school  at  Baylor  and  headed  the  artificial 
heart  program. 

Characterization  of  the  Task  Based  on  Interview  1 - We  rated  the 

following  characteristics  as  essential  to  surgery:  spacial 

reasoning,  temporal  reasoning,  means-ends  analysis,  formation  of 
sub-goals,  procedures,  decomposition,  visual  input,  verbal  input, 
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kinesthetic  input,  written  input,  instrumentation,  historical  data,  a 
large  branching  factor,  complex-  continuous  motor  movements, 
simple-discrete  motor  movements,  fine  motor  control,  gross  motor 
control,  repetition,  categorization,  learning,  analysis,  deduction, 
choosing,  planning,  supervising,  monitoring,  interpreting,  recall  of 
facts,  principles,  and  procedures,  identifying  objects,  actions,  and 
events,  stress,  precision,  cognitive  attentiveness,  attention  span, 
work  schedule,  high  perceived  probability  of  success,  instruments, 
notation,  directing,  supervising.  According  to  a pairwise  comparison 
of  the  raw  data,  surgery  is  most  similar  to  medical  diagnosis  (87) 
and  least  similar  to  DRAIR  generation  (54).  A cluster  including 
surgery  and  pilot  training  emerged  from  the  eight-cluster  analysis. 
They  had  high  correspondence  on  the  following  characteristics: 
spatial  reasoning,  temporal  reasoning,  mental  modeling,  formation  of 
sub-goals,  procedures,  and  specialization. 

Approach  - It  would  be  difficult  to  observe  the  task  actually  being 
performed  because  of  the  limited  access  to  operating  rooms.  However, 
we  could  have  the  expert  describe  a particular  operation,  such  as 
removal  of  the  appendix.  The  expert  can  use  visual  aids,  such  as  a 
diagram  of  the  human  body,  to  discuss  the  various  decisions  and 
procedures . 

Justifications  - This  task  is  very  visual  and  procedural  in  nature. 
Discussion  of  a specific  surgical  procedure  should  help  to  illuminate 
the  process  that  is  used  in  general. 

Results  - We  were  lucky  enough  to  have  the  opportunity  to  actually 
observe  a surgical  procedure  in  the  operating  facilities  at  SwRI. 
Dr.  Hall  had  to  perform  an  operation  to  insert  a catheter  into  the 
Illiac  artery  of  a baboon.  We  were  able  to  observe  preparation  of 
the  surgeon  and  the  surgical  team,  though  not  the  preparation  of  the 
operating  room  or  the  patient.  Dr.  Hall  talked  through  the  process 
and  showed  us  the  anatomy  and  what  he  was  doing  at  each  step.  From 
the  observation  it  became  quite  clear  that  surgery  depends  on  many 
other  subskills  from  the  ability  to  recognize  an  artery  vs.  a vein 
vs.  a nerve,  to  the  ability  to  recognize  and  describe  the  various 
instruments  used  in  the  surgical  process.  To  the  untrained  eye  they 
all  look  pretty  much  the  same.  In  addition,  the  procedure  hinges  on 
a complete  understanding  of  the  anatomy  involved  and  an  ability  to 
palpate  to  determine  where  you  are  in  the  body. 

7.  Medical  Diagnosis 

Brief  Task  Description  - Primary  care  medical  diagnosis  involves 
collecting  a complete  patient  history,  performing  a physical 
examination,  and  using  the  data  collect  to  categorize  the  symptoms 
into  a disease  category.  He  feels  that  most  of  his  diagnosis  is 
based  on  pattern  recognition.  He  recognizes  a gestalt  of  symptoms 
that  is  representative  of  a disease  category.  In  fact,  he's  willing 
to  allow  some  symptoms  to  remain  unexplained  by  the  diagnosis  as  long 
as  they  are  unimportant.  And  he  uses  time  as  a diagnostic  tool;  the 
pattern  of  the  symptoms  are  just  as  important  as  their  current 
presentation.  The  challenge  is  accumulating  the  data  and  grouping  it 
into  diagnostic  patterns. 
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Expert  Characteristics  - Medical  doctors  are  highly  trained  both  in 
formal  education  and  in  an  extensive  appreticeship  period.  They 
usually  obtain  a biology  or  chemistry  bachelor's  degree,  and  then 
enter  medical  school  for  four  years.  During  that  four  years, 
students  perform  rotations  that  expose  them  to  the  various  areas  of 
medical  care.  They  actually  perform  the  tasks  of  a phsysician,  with 
a physician  watching  and  verifying  their  performance. 

Source  of  Expertise: 

Dr.  James  D.  Legler 

Assistant  Professor 

Department  of  Family  Practice 

University  of  Texas  Health  Sciences  Center 

San  Antonio,  TX 

(512)  270-3934 

Characterization  of  the  Task  Based  on  Interview  1 - We  rated  the 

following  characteristics  as  essential  to  medical  diagnosis: 
temporal  reasoning,  deduction,  propose  and  refine,  acquire  and 
present,  visual  input,  verbal  input,  written  input,  instrumentation, 
historical  data,  large  branching  factor,  categorization,  learning, 
analysis,  choosing,  interpreting,  recalling  facts,  principles,  and 
procedures,  a perceived  probability  of  success,  notation,  advising, 
answering,  communicating,  directing,  and  informing.  According  to  a 
pairwise  comparison,  medical  diagnosis  was  most  similar  to  protocol 
design  (88)  and  least  similar  to  air  traffic  control.  In  the 
eight-cluster  analysis,  medical  diagnosis  emerged  as  a singleton 
cluster. 


Approach  - Medical  diagnosis  can  be  broken  down  into  a number  of 
different  areas:  anatomy/physiology,  patient  history/interview, 
signs/symptoms  associated  with  diseases,  and  treatment  plan.  It 
would  be  difficult  to  observe  the  task  actually  being  performed 
because  of  the  limited  access  to  medical  records  of  a patient. 
However,  we  could  have  the  expert  describe  a particular  case,  based 
on  a paitent's  file  where  the  name  has  been  deleted. 


Justifications  - Going  through  an  example  case  would  allow  us  to 
observe  the  actual  process  of  performing  a patient  diagnosis.  When 
it  is  not  in  an  actual  situation,  we  would  also  be  able  to  ask 
questions  and  have  the  benefit  of  hindsight.  This  hindsight  should 
be  useful  in  earlier  interviews.  Later  interviews  would  probably  be 
better  handled  in  an  actual,  on-the-spot  performance,  in  order  to 
avoid  that  hindsight. 

Results  - Dr.  Legler  pulled  a file  of  a 20-month  old  patient  that  he 
was  called  in  on  at  the  time  of  her  birth.  He  selected  this  file 
because  it  was  an  unusual  case  and  many  of  the  paitent's  problems 
have  still  not  been  resolved.  Ue  went  through  the  patient  record 
visit  by  visit,  discussing  the  examination  and  the  outcome.  Based  on 
this  interview,  we  developed  a fairly  clear  understanding  of  the 
process,  but  more  detail  on  actual  history  taking  and  linking  of 
signs  and  symptoms  to  problems  would  be  necessary  in  order  to  build  a 
tutoring  system. 
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TYPE  B TASKS 
1 • Software  Design 

Brief  Task  Description  - From  Susan  Crumrine's  perspective,  one  of 
the  major  tasks  associated  with  software  design  is  the  definition 
study.  This  study  specifies  the  purpose  of  the  system  and  determines 
the  functionality  it  must  achieve  to  meet  the  purpose.  The  output  of 
this  process  is  a conceputal  design  of  the  system. 

The  definition  study  answers  nine  questions:  1)  the  purpose  of  the 

system;  2)  the  functionality  requirements;  3)  the  sequence  of 
operations  necessary  to  perform  the  task;  4)  the  components  of  the 
system/organization  of  the  system;  5)  external  interfaces;  6) 
characteristics  of  the  user  interface;  7)  fault  tolerances;  8)  error 
detection/self  diagnostics;  and  9)  any  environmental  considerations. 

During  the  design  phase,  you  must  also  specify  which  parts  of  the 
functionality  will  be  covered  by  commercial,  off-the-shelf  (COTS) 
products  and  which  parts  will  need  custom  software  to  complete.  In 
addition,  the  design  phase  of  software  development  requires  enormous 
input  from  the  customer.  It  is  at  this  point  that  the  developers 
must  communicate  with  the  targeted  users  to  ensure  that  the  product 
will  reflect  their  needs. 

Expert  Characteristics  - Individuals  who  are  trained  in  performing 
software  design  are  usually  degreed  computer  scientists.  These 
individuals  could  have  a bachelor's,  Masters',  or  Ph.D.  in  the 
field.  However,  the  various  steps  performed  in  software  design  in 
the  real  world  are  only  trained  generally,  if  at  all,  at  a university 
or  college.  True  expertise  is  gained  on  the  job  working  for  someone 
who  is  already  skilled  at  the  task.  Often,  the  various  stages  that 
lead  to  an  implementation  of  a software  system  are  performed  by 
different  individuals  with  different  skills.  More  design-oriented 
tasks  tend  to  be  performed  by  the  better-degreed,  more-skilled 
individuals  while  coding  is  left  to  the  less-degreed, 
less-experienced  individuals. 

Source  of  Expertise: 

Susan  Crumrine 

Manager,  Software  Engineering  Section 
Southwest  Research  Institute 
San  Antonio,  TX 
(512)  522-2089 

Characterization  of  the  Task  Based  on  Interview  1 - We  rated  the 

following  characteristics  as  essential  to  software  design:  temporal 

reasoning,  mean-ends  analysis,  formation  of  sub-goals,  generate  and 
test  strategy,  propose  and  refine  strategy,  acquire  and  present  data, 
decompose  the  problem,  recombine  the  sub-problems,  verbal  inputs, 
written  inputs,  translation,  analysis,  integration,  computers, 
notation,  and  negotiation.  Based  on  a pairwise  comparison  of  the 
data,  software  design  emerged  as  most  similar  to  protocol  design  (96) 
and  least  similar  to  pilottraining  (54).  This  was  confirmed  in  the 
cluster  analysis:  protocol  design  and  software  design  emerged  as  a 
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cluster.  The  cluster  was  based  on  similarity  on  the  following 
characteristics:  temporal  reasoning,  analogical  reasoning,  deductive 

reasoning,  generation  of  sub-goals,  generate  and  test  strategy,  goal 
drive  problem  solving,  lack  of  cover  and  differentiate  strategy, 
propose  and  refine  strategy,  acquire  and  present  data,  and 
decomposition. 

Approach  - Software  design  can  be  broken  down  into  several  major 
subgoals  including  requirements  specification,  system  design,  system 
coding,  and  system  testing.  We  first  need  to  pick  one  of  these 
subgoals  and  concentrate  on  it  for  the  second  interview.  Follow-on 
interviews  may  continue  to  select  sub-subgoals,  etc. 

Justifications  - Software  design  is  a very  large,  complex  task.  In 
order  to  effectively  perform  knowledge  acquisition,  the  task  must  be 
broken  down  into  its  component  parts  and  examined  independently. 

Results  - We  gave  Susan  the  task  of  describing  how  the  design  of  a 
software  system  is  generated.  She  began  by  explaining  the  steps 
involved  with  design  at  any  level:  1)  break  the  system  down  into 

lower  level  funtions,  2)  identify  the  functions  of  each  subcomponent, 
and  3)  identify  the  interaction  between  subcomponents.  Task 
complexity  is  handled  by  breaking  into  subcomponents  and  defining 
interfaces  between  them  so  that  they  can  then  be  handled 

independently  as  needed.  The  real  key  and  difficulty  is  identifying 

logical,  useful  subcomponent  breakdowns.  This  appears  to  be  an 
ability  that  is  associated  with  generic  problem  solving  skills  and  is 
difficult  if  not  impossible  to  train.  This  second  interview  was  much 
more  successful  than  some  of  the  others  that  we  have  had  with  experts 
in  the  larger,  less-defined  tasks  because  Susan  has  spent  a lot  of 
time  thinking  about  how  to  do  the  task  and  how  to  get  others  to  do  it 
as  well. 

2.  Protocol  Design 

Task  Description  — Scientific  protocol  design  involves 
designing  an  experiment  to  test  a hypothesis.  It  involves  an 

assessment  of  ethical  and  legal  standards  as  well  as  the 

operationalization  of  the  hypothesis.  To  develop  a protocol,  the 
researchers  must  have  a clear  statment  of  the  objectives  and 
hypotheses.  In  addition,  they  must  set  up  the  constraints  that  have 
to  be  fulfilled.  Protocol  execution  is  a six  step  process:  1) 

assemble  resources;  2)  prepare  resources;  3)  use  resources;  4) 
analyse  results;  5)  draw  conclusions;  and  6)  report  the  conclusions. 

Expert  Characteristics  - Experts  in  the  field  of  protocol  design  tend 
to  have  engineering  or  scientific  degrees.  They  are  trained  in  the 
scientific  method  and  apply  it  to  the  development  of  protocols  for 
research  in  their  specific  area  of  interest. 

Source  of  Expertise: 

Herb  Peel 

Principle  Scientist  in  Biomedical  Engineering 
Southwest  Research  Institute 
San  Antonio,  TX 
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(512)  522-2692 

Herb  Peel  received  a B.S.  in  Engineering  at  the  University  of  Texas 
at  Austin.  He  worked  as  a clinical  engineer  at  a hospital  before 
coming  to  Southwest  Research  Institute  15  years  ago. 

Characterization  of  the  Task  Based  on  Interview  1 - We  rated  the 
following  characteristics  as  essential  to  protocol  design:  temporal 
reasoning,  case-based  reasoning,  developing  sub-goals,  generate  and 
test,  propose  and  refine,  acquire  and  present,  decomposition,  verbal 
inputs,  historical  inputs,  propadeutics,  categorization,  itemization, 
translation,  choosing,  planning,  interpreting,  procedures,  cases  and 
examples,  perceived  probability  of  success,  instruments,  and 
communication.  Protocol  design  was  most  similar  and  most  equivalent 
to  software  design.  Software  design  and  protocol  design  clustered  in 
the  cluster  analysis. 

Approach  - A particular  situation  should  be  generated  so  that  the 
expert  is  required  to  walk  through  the  process  of  generating  a 
protocol . 

Justifications  - Protocol  design  is  "case-based"  and  utilizes 
"propose  and  refine"  heavily.  We  needed  a way  of  looking  at  how  the 
initial  case  is  selected/proposed  and  how  it  is  refined  to  fit  the 
current  problem. 

Results  - The  situation  selected  was  to  pretend  that  we  were  a 
potential  client,  and  that  we  represent  a company  that  makes  and 
markets  an  automated  blood  pressure  monitor  (ABPM).  Herb  was  to 
develop  a protocol  that  he  would  use  to  compare  our  ABPM  against  two 
other  competitor's  ABPMs  and  a gold  standard.  We  did  not  give  him 
the  problem  in  advance  because  we  wanted  him  to  solve  the  problem 
while  we  were  there,  not  retrospectively.  Herb  basically  stuck  to 
the  specific  example.  Giving  him  a situation  resulted  in  learning 
right  away  that  a standard  existed  to  cover  the  situation.  Thus,  we 
discussed  the  standard  and  how/why  it  would  be  modified  to  fit  our 
particular  scenario. 

3.  Program  Management 

Brief  Task  Description  - Program  management  within  the  Air  Force 
involves  a knowledge  of  how  acquisitions  are  performed  and  a 
knowledge  of  the  procurement  process.  Within  the  Logistics  Command, 
no  formal  methodology  exists  and  much  of  the  process  has  been 
borrowed  from  Systems  Command  and  customized.  The  standard  series  of 
steps  that  must  be  followed,  usually  involving  the  generation  of  a 
document,  can  be  tailored  further  for  a particular  procurement.  A 
program  manager  must  monitor  the  progress  of  a particular  activity 
and  help  to  ensure  that  the  acquisition  goes  along  on  schedule.  When 
it  becomes  difficult  to  keep  to  the  original  schedule,  then  decisions 
have  to  be  made  concerning  how  to  modify  the  schedule  and  the  impact 
of  these  modifications  on  the  other  parts  of  the  acquisition. 

Expert  Characteristics  - Program  managers  receive  no  formal  training. 
They  learn  how  to  perform  the  required  tasks  on-the-job  and  with  the 
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help  of  more  experienced  program  managers  in  their  office.  The  task 
requires  the  ability  to  write  and  to  deal  with  people. 

Source  of  Expertise: 

Mr.  Xavier  Pena,  Program  Manager 
Air  Force  Air  Logistics  Command/LDAT 
Kelly  Air  Force  Base,  Texas 
(512)  925-4714 

Characterization  of  the  Task  Based  on  Interview  1 - Program 
management  received  a rating  of  four  in  the  following  attributes: 
temporal,  sub-goals,  verbal,  written,  historical,  plan,  supervise, 
monitor,  procedures,  communicates,  and  negotiates.  The  task  received 
a rating  of  zero  in  the  following  attributes:  statistical,  spatial, 

analogical,  model-based,  mathematical,  deductive,  inductive, 

means-ends,  generate-and-test , cover-and-differentiate,  recombine, 
generalize,  specialize,  visual,  auditory,  kinesthetic, 
instrumentation,  propadeutics , complex-continuous,  compound,  reflex, 
simple-discrete,  fine,  gross,  repetitive,  calculate,  code, 
computerize,  interpolate,  tabulate,  translate,  deduce,  induce, 
compute,  integrate,  analogs,  perceptual-speed, 

search- receive- informat  ion,  identify-objects-actions-events, 

scan-display,  acceleration,  confinement,  isolation,  contaminants, 
electricity,  lighting,  magnetism,  noise,  fatigue,  mental-strain, 
stress,  physical-strain,  preciseness,  cognitive-attentiveness, 

response-chaining,  attention-span,  sleep,  schedule,  boredom, 
general-health,  age,  gender,  height,  weight,  introspective-accuracy, 
articulation,  vehicles,  weapon-systems,  instruments,  notation, 
test-equipment,  advises,  indicates,  instructs,  expressive-movement, 
and  interpretive-movement.  According  to  a pairwise  comparison, 

program  management  is  most  similar  to  to  leadership  (99)  and  least 
similar  to  aircraft  pilot  (57).  It  clustered  with  cargo  loading, 
accounting,  leadership,  equipment  diagnosis,  DRAIR  generation,  and 
form  fill-out  in  the  eight-cluster  analysis. 

Approach  - Program  Management  can  be  broken  down  into  fairly 
independent  subtasks,  such  as  statement  of  need  (SON),  statement  of 
work  (SOU),  request  for  proposal  (RFP) , etc.  Each  of  these  subtasks 
is  an  "acquire  and  present"  task,  so  they  are  very  similar.  We  need 
to  acquire  the  general  set  of  subtasks,  and  then  select  one  for 
further  study. 

Justifications  - Program  management  is  a very  large,  ill-defined 
task.  However,  it  has  many  subtasks  associated  with  it  and  the 
knowledge  acquisition  process  can  be  simplified  if  these  subtasks  can 
be  identified  and  then  further  studied  independently. 

Results  - We  began  the  interview  by  discussing  potential  documents 
that  they  need  to  generate  in  order  to  perform  their  task.  However, 
it  appears  that  most  of  the  documents  are  actually  delivered  under 
contract  or  by  the  governing  command.  The  Logistics  Command  has  the 
problem  of  being  handed  programs  that  are  already  underway  that  they 
then  have  to  pick  up  and  manage.  After  discussing  other  tasks  such 
as  review  of  progress  reports  and  presentation  of  programs  to 
management,  we  got  into  the  funding  document.  From  that  discussion 
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we  were  able  to  learn  something  about  how  money  is  programmed  and  the 
types  of  money  that  exist  for  the  various  functions  within  AFLC. 
However,  more  discussions  would  be  required  in  order  to  obtain  a 
sufficient  understanding  of  this  process. 

4.  Leadership  Training 

Brief  Task  Description  - We  examined  the  total  quality  management 
(TQM)  initiative  in  relation  to  leadership  training.  In  this 
initiative,  students  learn  the  basic  tools  and  techniques  used  to 
ensure  the  sucess  of  participative  management  through  team  building. 
The  students  learn  about  how  to  develop  process  action  teams  that 
focus  on  improving  an  existing  management  or  technical  function. 

Expert  Characteristics  - Individuals  involved  with  leadership 
training  are  usually  involved  with  management  in  general.  They  tend 
to  have  at  least  a Bachelor's  degree.  Little  other  formal  training 
is  provided,  except  for  some  on-the-job  experiences. 

Source  of  Expertise: 

Jim  Shawley 

Wright  Patterson  AFB,  Ohio 
(513)  257-1602  (3931) 

Jim  Shawley  worked  as  a management  analyst  for  the  Foreign  Technology 
Division  at  WPAFB  for  32  years.  He  now  works  in  the  Total  Quality 
Management  office. 

Characterization  of  the  Task  Based  on  Interview  1 - We  rated  the 
following  characteristics  as  essential  for  leadership  training: 
verbal  inputs,  analysis,  supervision,  procedures,  introspective 
accuracy,  articulation,  advising,  communication,  informing,  and 
instructing.  According  to  a pairwise  comparison,  leadership  training 
was  most  similar  to  program  management  (99)  and  least  similar  to 
pilot  training  (61).  In  the  eight-cluster  analysis,  leadership 
training  clustered  with  cargo  loading,  accounting,  program 
management,  equipment  diagnosis,  drair  generation,  and  form  fill-out. 

Approach  - Leadership  training  in  the  form  that  we  are  examining 
involves  the  area  of  Total  Quality  Maintenance  (TQM).  TQM  consists 
of  a set  of  tools  and  procedures  for  data  collection  and  application 
of  the  tools.  We  should  identify  the  set  of  tools  and  select  a 
single  tool  for  further  examination. 

Justifications  - Leadership  training  is  a fairly  nebulous  task  hut  it 
consists  of  using  many  different  tools.  Learning  about  the  tools  and 
how  they  are  applied,  one  at  a time,  will  help  to  better  define  the 
goals  of  the  knowledge  acquisition  process. 

Results  - We  examined  the  Student  Guide  for  a three  day  teambuilding 
course  that  Jim  was  currently  teaching.  Teambuilding  is  one  aspect 
of  the  leadership  training  that  is  provided  at  Wright  Patterson. 
This  teambuilding  concept  can  be  used  to  support  a variety  of  the 
teams  that  may  be  built  based  on  the  TQM  philosophy  (ie.  PAT,  PIT, 
etc.).  The  Student  Guide  provided  an  excellent  structure  to  the 
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discussion  of  a very  nebulous  and  general  technique.  One  of  the  key 
tools  discussed  was  brainstorming  and  how  it  should  be  performed 
within  the  teambuilding  concept.  Fishbone  diagrams,  Pareto  diagrams, 
and  control  charts  were  also  discussed  based  on  their  use  in  support 
of  brainstorming  and  recording  the  results. 

5.  Cargo  Loading 

Brief  Task  Description  - Cargo  loading  involves  the  determining  where 
to  place  a given  set  of  items  in  the  hold  of  a specific  aircraft. 
The  task  involves  a knowledge  of  what  types  of  items  can  go  next  to 
each  other,  how  the  various  types  of  items  must  be  secured,  the 
weigths  and  packaging  of  the  various  items,  and  to  balance  the  load 
for  the  given  aircraft.  Formulas  exist  for  the  calculations,  but 
experience  is  often  used  as  a start  point  for  arranging  the  items. 

Expert  Characteristics  - Individuals  are  trained  to  perform  the  task, 
but  a background  in  basic  arithmetic  is  necessary  in  order  to  learn 
how  to  peform  the  various  calculations.  The  individuals  do  not  tend 
to  have  a college  degree.  The  heuristics  for  knowing  how  to  combine 
and  place  loads  come  through  experience  and  practice. 

Source  of  Expertise: 

Senior  Master  Sgt.  Ronald  Whitmer 

Building  901 

Randolph  Air  Force  Base,  Texas 

(512)  652-2030 

Characterization  of  the  Task  Based  on  Interview  1 - Cargo  loading 

received  a rating  of  four  in  the  following  attributes:  spatial, 

temporal,  mathematical,  sub-goals,  formula-procedure-algorithm, 
written,  calculate,  itemize,  compute,  estimate,  plan,  monitor,  facts, 
principles,  and  supervises.  The  task  received  a rating  of  zero  in 
the  following  attributes:  statistical,  analogical,  inductive, 

generate-and-test , cover-and-dif ferentiate,  propose-and-refine, 
generalize,  verbal,  auditory,  kinesthetic,  instrumentation, 
man-machine-interface,  databases,  historical,  uncertainty, 

complex-continuous,  compound,  reflex,  simple-discrete,  fine,  gross, 
repetitive,  code,  computerize,  tabulate,  translate,  induce, 
integrate,  interpret,  analogs,  perceptual-speed, 

search-receive-information,  identify-objects-actions-events, 

scan-display,  isolation,  contaminants,  electricity,  lighting, 
magnetism,  fatigue,  mental-strain,  stress,  preciseness, 
cognitive-attentiveness,  response-chaining,  attention-span,  sleep, 
schedule,  age,  gender,  height,  weight,  introspective-accuracy, 
probability-of-success,  computer,  weapon-systems,  instruments, 

test-equipment,  advises,  indicates,  informs,  instructs,  transmits, 
negotiates,  and  interpretive-movement.  According  to  a pairwise 
comparison,  cargo  loading  is  most  similar  to  to  program  management 
(92)  and  least  similar  to  aircraft  pilot  (56).  It  clustered  with 
accounting,  program  management,  leadership,  equipment  diagnosis, 
DRAIR  generation,  and  form  fill-out  in  the  eight-cluster  analysis. 

Approach  - Cargo  loading  involves  the  use  of  a number  of  formulas  and 
calculations,  as  well  as  heuristics  gained  from  experience.  What 
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exactly  is  taken  into  consideration  in  planning  a cargo  is  dependent 
on  the  cargo  and  the  aircraft.  We  should  then  watch  the  expert 
perform  the  task  for  a single  aircraft  and  in  increasingly  more 
complex  set  of  cargo,  discussing  the  process  as  he  goes  along. 

Justifications  - Cargo  loading  is  very  procedural,  but  variations 
occur  due  to  the  type  of  aircraft  and  type  of  cargo.  Thus,  we  should 
begin  to  investigate  the  knowledge  by  selecting  one  area  and  learning 
the  procedures  as  they  apply  to  that  selected  area,  then  move  on  to 
other  areas. 

Results  - SMSgt.  Whitmer  was  prepared  with  one  of  the  training  books 
used  to  teach  cargo  loading.  He  began  with  a discussion  of  certain 
terminology,  which  required  diagramming  an  aircraft.  He  explained 
what  all  of  the  terms  meant,  why  they  were  needed,  and  how  to 
generate  the  input  data  to  the  problem.  We  then  went  through  the 
formulas  used  to  evaluate  a load  plan.  Basically,  these  formulas  are 
used  to  verify  that  the  planned  load  meets  requirements.  Then  we 
went  through  the  process  of  planning  a load  for  a simple  case,  namely 
an  all-pallet  load.  Then  we  moved  into  how  vehicles  are  calculated 
in  to  the  load  and  planned  for.  The  interview  provided  an  enormous 
amount  of  specific  information  that  could  be  used  in  teaching  this 
task. 

6.  Equipment  Diagnosis 

Brief  Task.  Description  - Diagnosis  and  repair  of  complex  equipment 
requires  a variety  of  support  equipment  and  documentation.  In 
diagnosing  problems  with  aircraft  engines,  automatic  test  equipment 
provides  a lot  of  data  concerning  the  functioning  and  non-functioning 
of  the  problem  engine.  In  addition,  work  control  documents  provide 
the  general  procedures  to  follow  for  first  diagnosing,  and  then 
repairing  an  identified  problem.  A history  is  kept  on  each  engine  so 
that  the  individual  who  must  fix  it  knows  when  it  was  in  last,  what 
the  problems  were,  and  how  these  problems  were  resolved.  Though  all 
of  this  support  is  available  to  the  individual  faced  with  fixing  the 
engine,  the  task  still  requires  experience  in  order  to  be  effective. 

Expert  Characteristics  - Experts  are  not  necessarily  degreed  or  even 
provided  formal  training  in  the  diagnosis  and  repair  of  the  items 
they  are  responsible  for.  They  work  their  way  up  through  the  ranks 
based  on  their  experience,  gained  primarily  on  the  job.  The  job  can 
become  repetitive,  and  there  is  often  pressure  from  upper  level 
management  to  produce  at  a more  rapid  rate  than  is  possible. 

Source  of  Expertise: 

Arthur  Martinez 

Air  Force  Air  Logistics  Command/ 

Kelly  Air  Force  Base,  Texas 

(512)925-8757 

Characterization  of  the  Task  Based  on  Interview  1 - Equipment 
diagnosis  received  a rating  of  four  in  the  following  attributes: 
formula-procedure-algorithm,  instruments,  test-equipment,  repetitive, 
compare,  procedures,  and  noise.  The  task  received  a rating  of  zero 
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in  the  following  attributes:  statistical,  analogical,  mathematical, 
inductive,  means-ends,  cover-and-di f ferentiate,  propose-and-ref inej 
generalize,  . specialize,  verbal,  kinesthetic,  dynamism, 
complex-continuous,  compound,  reflex,  simple-discrete,  code, 
computerize,  interpolate,  tabulate,  translate,  induce,  choose, 
compute,  estimate,  integrate,  plan,  supervise,  monitor,  analogs, 
perceptual-speed , search-receive-information, 
identify-objects-actions-events,  scan-display,  acceleration, 
confinement,  isolation,  lighting,  magnetism,  fatigue,  mental-strain, 
stress,  preciseness,  cognitive-attentiveness,  response-chaining, 
sleep,  age,  gender,  height,  weight,  introspective-accuracy, 
probability-of-success,  articulation,  vehicles,  weapon-systems, 
notation,  advises,  answers,  communicates,  directs,  indicates, 
informs,  instructs,  requests,  transmits,  supervises,  negotiates, 
expressive-movement,  and  interpretive-movement.  According  to  a 
pairwise  comparison,  equipment  diagnosis  is  most  similar  to  to 
accounting  (88)  and  least  similar  to  aircraft  pilot  (56).  It 
clustered  with  cargo  loading,  accounting,  program  management, 
leadership,  DRAIR  generation,  and  form  fill-out  in  the  eight-cluster 
analysis. 

Approach  - Ideally,  we  should  follow  a specific  engine  from  start  to 
finish  through  the  repair  process.  However,  that  can  take  a period 
of  a month  or  more,  and  we  will  only  obtain  information  on  the 
problems  that  that  particular  engine  had.  Because  an  attempt  has 
been  made  to  proceduralize  the  task  through  the  use  of  technical 
orders  and  test  equipment,  an  example  technical  order  should  be 
examined. 

Justifications  - Examination  of  the  technical  order  will  provide  a 
basis  for  understanding  the  processes  and  procedures  that  are 
followed  during  a repair.  However,  it  is  important  to  also  learn 
where  they  stop  using  the  technical  orders  and  why.  For  example,  the 
technical  orders  may  not  be  up-to-date  or  they  have  discovered  a more 
efficient  approach  based  on  experience. 

Results  - Mr.  Martinez  was  prepared  with  a couple  of  example 
technical  orders  (TOs)  which  we  examined  during  the  interview.  These 
TOs  contain  the  step-by-step  procedure,  including  drawings,  in  order 
to  perform  any  task  that  may  appear  on  the  work  control  document 
(WCD).  It  became  apparent  in  this  second  interview  that  to  truly 
interview  someone  in  diagnosis  we  need  to  talk  with  members  of  the 
On-Condition  Maintenance  (OCM)  team.  The  area  where  Mr.  Martinez 
works  performs  the  actual  repair  but  does  not  actually  do  the 
diagnosis.  The  diagnosis  is  handled  by  OCM  team  members  through  an 
inspection,  and  the  results  are  placed  on  a WCD  for  use  by 
individuals  in  the  shop  in  repairing  a given  item.  The  TOs  provide 
the  information  on  how  to  perform  the  repairs. 

7.  DRAIR  Generation 

Brief  Task  Description  - DRAIR  generation  is  a databased-decision 
making  task.  The  process  of  generating  a Deficiency  Report  Analysis 
Information  Report  (DRAIR)  requires  the  acquisition  of  a large  amount 
of  data  that  must  then  be  interpreted  with  respect  to  maintenance 
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times,  support  costs,  etc.  A DRAIR  is  generated  based  on  an  inquiry 
from  an  equipment  specialist,  item  manager,  or  other  individual  who 
suspects  that  a problem  might  exist  with  the  reliability  of  a 
particular  item.  The  result  is  a written  document  that  describes  the 
findings  of  the  data  analysis  and  provides  an  assessment  of  whether 
or  not  the  data  indicates  a material  deficiency,  and  if  so,  what 
should  be  done  about  it. 

Expert  Characteristics  - Experts  have  primarily  an  operations 
research  degree.  They  are  not  formally  trained  in  the  generation  of 
DRAIRs  but  learn  by  doing  the  task  on-the-job.  An  individual  is 
usually  assigned  a particular  weapon  system  or  set  of  weapon  systems 
for  which  they  gain  more  detailed  expertise  based  on  generating 
DRAIRs.  The  individuals  do  not  necessarily  have  much  expertise  in 
the  engineering  areas  of  the  items  themselves,  such  as  hydraulics  or 
electronics. 

Source  of  Expertise: 

Gayle  Davis 

Air  Force  Air  Logistics  Command/TIETP 

Tinker  Air  Force  Base,  Oklahoma 
(405)  736-5015  (5034) 

Characterization  of  the  Task  Based  on  Interview  1 - DRAIR  generation 
received  a rating  of  four  in  the  following  attributes:  inductive, 
acquire-and-present , verbal,  databases,  historical,  itemize,  analyze, 
induce,  integrate,  interpret,  facts,  articulation,  computer,  answers, 
communicates,  and  informs.  The  task  received  a rating  of  zero  in  the 
following  attributes:  spatial,  model-based,  means-ends, 
generate-and-test , cover-and-dif ferentiate,  propose-and-ref ine, 
specialize,  return-to-def ini tion,  visual,  auditory,  kinesthetic, 
instrumentation,  man-machine-interface,  propadeutics , branching, 
dynamism,  constraints,  complex-continuous,  compound,  reflex, 
simple-discrete,  fine,  gross,  repetitive,  calculate,  code, 
computerize,  choose,  compute,  plan,  supervise,  monitor,  principles, 
perceptual-speed , search-receive-information, 
identify-objects-actions-events,  scan-display,  acceleration, 
confinement,  isolation,  contaminants,  electricity,  lighting, 
magnetism,  noise,  fatigue,  mental-strain,  stress,  preciseness, 
cognitive-attentiveness,  response-chaining,  attention-span,  sleep, 
schedule,  boredom,  general-health,  age,  gender,  height,  weight, 
introspective-accuracy,  probability-of-success,  vehicles, 
weapon-systems,  instruments,  test-equipment,  advises,  directs, 
indicates,  instructs,  supervises,  negotiates,  expressive-movement, 
and  interpretive-movement.  According  to  a pairwise  comparison,  DRAIR 
generation  is  most  similar  to  to  accounting  (101)  and  least  similar 
to  aircraft  pilot  (51).  It  clustered  with  cargo  loading,  accounting, 
program  management,  leadership,  equipment  diagnosis,  and  form 
fill-out  in  the  eight-cluster  analysis. 

Approach  - Generation  of  a DRAIR  can  be  broken  down  by  the  sources  of 
data  and  what  that  data  means.  We  should  select  one  set  of  data  and 
concentrate  on  how  it  is  obtained,  what  it  means,  and  it  is 
interpreted  and  summarized  in  the  preparation  of  a DRAIR.  This 
should  be  done  using  an  example  of  the  data  that  has  been  extracted 
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from  the  appropriate  data  system  and  the  resulting  DRAIR. 

Justifications  - An  example  is  always  useful  to  help  provide 
structure  and  direction  to  an  interview.  However,  simply  going 
through  an  example  with  no  discussion  of  the  generalities  behind  the 
decisions  and  interpretations  would  not  be  very  enlightening  at  this 
point  in  the  knowledge  engineering  process.  The  experts  use 
generalizations  and  it  is  more  efficient  to  obtain  those  directly, 
rather  than  trying  to  generalize  from  a bunch  of  examples. 

Results  - Gayle  Davis  was  prepared  for  the  interview  with  example 
SAFE  and  CPA  reports,  plus  documents  on  the  data  systems  that  are 
used  in  her  job.  We  examined  the  data  reports  and  discussed  the 
useful  data  points,  where  they  come  from,  what  they  mean,  and  how 
decisions  are  made  based  on  the  data  points.  These  decisions  are 
then  documented  in  the  DRAIR.  the  structure  of  the  DRAIR  report 
itself  was  also  discussed  briefly  and  how  the  data  feeds  the  various 
components.  Based  on  this  interview,  we  were  able  to  obtain 
considerable  information  concerning  the  data  to  look  for,  where  to 
look  for  it,  and  what  to  say  based  on  it  in  the  generation  of  a 
DRAIR. 

8.  Form  Fill-Out 

Brief  Task  Description  - Our  initial  interview  with  Bill  Bayliss 
covered  the  basic  task  of  generating  a cost  proposal  for  submission 
to  a DOD  agency.  He  discussed  the  data  in  an  RFP  that  drives  the 
contents  of  the  cost  proposal,  including  the  type  of  contract  and  the 
FAR  clauses.  From  this  interview,  we  targeted  a specific  form  he 
completes  to  specify  all  documentation  included  in  the  cost  proposal. 

Expert  Characteristics  - Form  fill-out  is  a very  broad  topic  and 
could  be  subsumed  under  many  other  tasks.  Thus,  expert 

characteristics  can  vary  significantly,  depending  on  the  type  and 
purpose  of  the  form. 

Source  of  Expertise: 

Bill  Bayliss 
Contract  Administrator 
Southwest  Research  Institute 
San  Antonio,  TX 
(512)  522-2238 

Bill  Bayliss  is  a contract  administrator  at  Southwest  Research 
Institute.  He  has  a background  in  finance  of  over  40  years 
experience  in  addition  to  a masters  degree  in  finance  and  accounting. 
Prior  to  working  at  SwRI,  he  was  a contract  administrator  for  the  Air 
Force. 

Characterization  of  the  Task  Based  on  Interview  1 - We  rated  the 

following  characteristics  as  essential  to  form  fill-out:  generate 
sub-goals,  acquire  and  present  data,  written  inputs,  making  choices, 
procedures,  use  of  a computer,  transmitting  information,  and 

negotiation.  According  to  a pairwise  comparison,  form  fill-out  is 
most  similar  to  leadership  training  (98)  and  least  similar  to  pilot 
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training  (54).  In  the  eight-cluster  analysis,  form  fill-out 
clustered  with  cargo  loading,  accounting,  leadership  training, 
equipment  diagnosis,  and  drair  generation,  based  primarily  on  the 
formation  of  sub-goals,  acquire  and  presentation  of  data,  and 
well-specified  procedures. 

Approach  - Form  fill-out  is  defined  by  the  form  itself  and  the 
background  knowledge  required  to  fill  it  out.  We  should  select  an 
example  form  and  the  data  sources  to  support  it,  and  watch  the  expert 
fill  the  form  out,  discussing  the  process  as  he  goes  along. 

Justifications  - The  task  is  basically  defined  by  the  form  to  be 
completed  which  provides  the  "procedure.”  It  is  an  "acquire  and 
present"  task  that  requires  the  current  situation  as  data  and  a broad 
background  of  general  knowledge  about  the  area  for  which  the  form  is 
used . 

Results  - We  selected  the  form  that  Bill  Bayliss  uses  to  outline  the 
construction  of  a contract  based  on  an  RFP.  We  took  the  ICATT  RFP 
that  he  had  worked  on  about  one  year  ago  and  followed  the  process 
required  by  the  form.  Bill  was  told  in  advance  about  the  problem, 
but  did  not  fill-out  the  form  in  advance.  Though  a specific  example 
was  discussed,  most  of  the  information  was  provided  in  generalities, 
with  rules  about  how  you  would  fill-in  each  slot.  The  specific 
example  wo'uld  not  have  lead  to  the  same  understanding  and  it  is 
apparent  that  even  after  many  specific  examples  it  would  be  difficult 
to  generalize  without  the  support  of  the  expert  in  making  those 
generalizations. 

9.  Accounting 

Brief  Task  Description  - In  the  initial  interview,  we  discussed  the 
variety  of  tasks  performed  by  an  accountant.  Accounting  is  very 
procedure  oriented;  the  books  must  be  in  compliance  with  the 
Generally  Accepted  Accounting  Principles  (GAP).  These  principles 
largely  establish  accounting  practice.  The  details  not  established 
by  GAP  are  institution  specific  and  established  on-site.  There  is 
some  ambiguity  in  interpretation  that  occassionally  must  be  resolved. 

Expert  Characteristics  - Experts  in  accounting  usually  have  at  least 
a Bachelor's  degree  with  a major  in  accounting.  Additional  tests  can 
be  taken  that  authorizes  the  individual  as  a Certified  Public 
Accountant.  However,  the  accounting  degree  usually  just  helps  the 
individual  to  understand  the  general  principles  behind  the  accounting 
task  at  a particular  institution.  Thus,  on-the-job  training  that 
familiarizes  the  individual  with  the  standard  practices  and 
procedures  of  the  specific  company  are  usually  required. 

Source  of  Expertise: 

Linda  Boehm,  CPA 
Accounting  Department 
Southwest  Research  Institute 
San  Antonio,  TX 
(512)  522-2965 
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Characterization  of  the  Task  Based  on  Interview  1 - We  rated  the 
following  characteristics  as  essential  for  accounting:  temporal 
reasoning,  case-based  reasoning,  formation  of  sub-goals,  generate  and 
test,  acquire  and  present,  decompose,  verbal  inputs,  historical 
inputs,  propadeutics,  categorization,  itemizing,  translating, 
choosing,  planning,  interpreting,  procedures,  use  of  past  cases, 
perceived  probability  of  success,  instruments,  and  communication. 
According  to  a pairwise  comparison,  accounting  was  most  similar  to 
DRAIR  generation  (101)  and  least  similar  to  pilot  training  (53).  It 
clusters  with  cargo  loading,  program  management,  leadership  training, 
equipment  diagnosis,  drair  generation  and  form  fill-out  in  the 
eight-cluster  analysis. 

Approach  - We  need  to  have  the  expert  select  a task  that  she  performs 
on  a regular  basis  that  takes  one  to  two  hours  and  watch  her  perform 
the  task,  discussing  the  process  as  she  goes  along. 

Justifications  - Accounting  consists  of  many  diverse  tasks  and  the 
tasks  vary  depending  on  the  individual's  job.  In  order  to  get  a 
handle  on  the  procedures  used  and  to  train  anything  useful,  the 
knowledge  acquisition  process  should  center  initially  on  a particular 
task. 

Results  - Linda  selected  the  task  of  reconciling  the  general  ledger 
with  the  projects  ledger.  The  task  is  performed  every  quarter  and  is 
used  to  verify  that  the  Institute's  financial  report  is  accurate. 
She  went  through  the  procedure,  which  basically  involved  filling  out 
a form,  while  we  asked  questions  concerning  where  she  was  getting  the 
data,  what  the  data  meant,  and  when  a problem  occurred  in  the 
balancing  where  she  tended  to  look  in  order  to  reconcile  it.  The 
process  is  highly  procedural  in  terms  of  filling  out  the  form,  but 
the  knowledge  concerning  where  and  how  to  look  for  problems  that 
occur  is  much  more  heuristically-based. 
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LIST  OF  KNOWLEDGE  ACQUISITION  TECHNIQUES  AND  TOOLS 

TRADITIONAL  METHODS 

A.  Written  Materials 
texts,  manuals,  etc. 

B.  Verbal  Reports 

1.  Interviews  - descriptive,  problem-oriented,  structured 

2.  Observational  Studies 
- discourse  analysis 

3.  Protocol  Methods  - retrospective  reports,  current  problem 

C.  Psychometric  Techniques 

1.  Personal  Constructs 

2.  Multidimensional  Scaling/Cluster  Analysis 

3.  Candidate  Elimination/Version  Spaces  in  concept  learning 
AUTOMATED  ACQUISITION  TOOLS 

A.  Induction-Based  Tools/Concept  Learning 

1.  RuleMaster 

2.  TIMM 

3.  Expert  Ease 

4.  ID3  - generates  decision  tree 

5.  PRISM  - generates  production  rules 

6.  MARVIN  - induces  and  represents  function  recursively 

7.  NODDY  - represents  function  in  nested  numeric  expressions 

8.  BACON  - clustering 

9.  COPER 

10.  LEX-II  - explanation-based  concept  learning 

11.  AM  - discovery-based  concept  learning 

12.  FOO  - learning  by  being  told 

13.  BRZDYL1  - a learning  system  for  conceptual  engineering  design 


B.  Interview-Based  Systems 

1.  THEIRESIAS  - ka  for  MYCIN 

2.  MOLE  - symptom/hypothesis  linkages  - successor  to  MORE 

3.  PLANET  - repertory  grid  methods 

4.  ROGET  - method  oriented  conceptual  model 

5.  ASK  - tool  for  acquiring  strategic  knowledge 

6.  ETS  - personal  constructs  used  to  build  concept  hierarchies 

7.  KNACK  - evaluation  - task-based  approach  for  advice  systems 

8.  SALT  - solution  construction  - proposed  and  revise 

9.  MORE  - model  of  classification  problem  solving 

10.  KITTEN  - another  rep  grid  tool 

11.  KREME  - for  developing  and  editing  large  kbs 

12.  ASTEK  - structured  ka  for  troubleshooting 

13.  SIZZLE  - ka  tool  for  the  sizing  task 

14.  RIME  - knowledge  acquisition  for  XCON 

15.  KAS  - ka  for  prospector  - semantic  networks 

16.  HYDRA  - augmented  transition  networks 

17.  AQUINAS  - successor  to  ETS 

18.  OPAL  - elicits  specific  cancer  treatment  plans 

19.  KADS  - develop  term  taxonomies/construct  knowledge-level  models 

20.  NEXTRA  - uses  repertory  grid  analysis 

21.  YAKA  - combination  of  qualitative  reasoning/cover  & differentiate 

22.  K"ac  - tool  for  knowledge  assimilation  given  existing  knowledge  base 

23.  SMEE  - personal  construct  elicitation 

24.  Tranowski’s  KAE  for  scene  analysis 

25.  FMS  Aid,  KSSO,  KRITON  - rep  grid  tools 

26.  TEST/TDE  - for  diagnostic  problem  solving 

COMMERCIAL  KNOWLEDGE  ACQUISITION  PRODUCTS 

A.  ADS  (Aion  Corporation) 

KA  module  within  expert  system 

B.  Knowledge  Shaper  (Perceptics,  Inc.) 

ID3/decision  tree  oriented  tool  for  KA 

C.  Socrates  (CIM  Solutions,  Inc.) 

KA  module  within  expert  system 

D.  Exsys  Professional  (Exsys,  Inc.) 

KA  module  within  expert  system 

E.  G-2  Real-time  Expert  (Gensym) 
object-oriented  rule  modeler 

F.  VP-Expert  (Paperback  Software,  Int’l) 

KA  module  within  expert  system 


G. 

Nextra  (Neuron  Data) 

Repertory  grid  based  KA  took  at  front  end  for  Nexpert  Object 

H. 

GURU  (mdbs  Inc.) 

KA  module  within  expert  system 

I. 

Mercury  KBE  (Artificial  Intelligence  Technologies) 
object  oriented/Lisp  based  KA  module 

J. 

KAM  (Phase-Linear,  Inc.) 

KA  from  text/documentation  parsing  service 

K. 

Knowledge  Analysis  Tool  (Gary  Ribar,  Inc.) 
modified  repertory  grid  based  KA  tool 

L. 

FOCUS  (Information  Builders) 
crude  KA  facsimile 

M. 

PIUS  (COGENSYS) 

captures  judgmental  expertise  without  knowledge  engineering 
Personal  Insurance  Underwriting  System 

N. 

CLS  (COGENSYS) 
same  as  above 
Consumer  Lending  System 

O. 

MUS  (GOGENSYS) 
same  as  above 

Mortgage  Underwriting  System 

P. 

TRILOGY  (Metavision,  Inc.) 

intelligent  tutoring  system;  automatically  constructs  courseware 

Q. 

1st  Class  (AlCorp  Inc.) 
induction,  hypertext 

R. 

Turbo  Shell  v3.0  (Berkshire  Software  Co.) 

creates,  modifies,  and  consults  knowledge  base,  uses  confidence  factors  and  fuzzy 
logic 

S. 

ALEX  (Harris  and  Hall  Associates)  ?? 
incremental  expert  system  development 

T. 

Level5  (Information  Builders,  Inc.) 

facilitates  the  transfer  of  expertise  into  usable  knowledge  bases 

| 

U. 

IXL  2.0  (Intelligence  Ware,  Inc)  • • 

generates  rules  from  a set  of  hypotheses  using  a proprietary  induction  algorithm 

1 
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£ 

| 

i 
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V. 

Autointelligence  (Intelligence  Ware,  Inc.) 

captures  and  distills  knowledge  of  an  expert  to  generate  an  expert  system 

w. 

Crystal  Induction  System  (Intelligent  Environments) 

generates  rules  from  a series  of  examples  entered  in  text  or  spreadsheet  form 

X. 

Sophos  (Cognisys  Teleport  de  Montreal) 
knowledge  acquisition  rule  generator 

Y. 

Knowledge  Quest  (Computer  Aided  Knowledge) 
knowledge  into  rules 

Z. 

MAINGEN  (OXKO  Corp.) 

up  to  100  rules  on  any  maintenance  task  with  fill-in-the  blank  Q and  A automated 
interview 

AA. 

INDUCPRL  (OXKO  Corp) 

takes  as  input  a spreadsheet  or  table  of  data  and  outputs  production  rules 

BB. 

LogicTree  (CAM  Software,  Inc.) 

uses  decision  trees  for  capturing  information 

CC. 

KnowledgeMaker  (Knowledge  Garden,  Inc.) 
induction-based 

APPENDIX  D 


FIRST  INTERVIEW  QUESTIONNAIRE 


Appendix  D 

Initial  Interview  Questions 


Initial  Discussion: 


There  are  no  right  answers  or  good  answers  to  these  questions.  Some  of  the  questions  mav 
e Phrased  so  that  you  believe  one  or  another  response  is  appropriate  or  somehow  better  than 
another.  For  example,  it  may  sound  good  to  say  you  plan  up  front  all  your  actions  in  solving  a 
pro  em  or  doing  the  job  even  if  some  of  those  actions  occur  somewhat  automatically  Please  do 
not  fall  into  this  bias.  For  this  example,  it  is  good  to  plan  but  it  is  also  good  not  to  plan  too  much 
and  to  have  some  actions  that  occur  automatically.  We  want  to  assess  all  aspects  of  your  iob  with 
as  much  fidelity  as  possible.  What  results  from  these  interviews  will  help  us  to  better  understand 
t e unique  aspects  of  your  work  and  how  we  could  learn  more  about  it  in  the  future  so  we  can 
design  computer  systems  that  help  you  perform  your  job  more  easily. 

Also,  we  want  to  assure  you  that  under  no  circumstances  will  your  answers  go  to  vour 
supervisor  or  anyone  else  outside  of  this  room.  You  are  responding  to  this  questionnaire 
completely  anonymously  Because  of  this,  we  want  you  to  provide  the  operational  answers  to  these 
questions,  not  the  official  ones.  We  want  to  know  what  you  really  do  on  your  job. 

A)  UNDERSTANDING  THE  PROBLEM 


Al)  Tell  us  a little  bit  about  your  field,  yourself,  and  how  you  got  into  this  field. 
**A2)  What  initiates  the  problem  solving  process? 

A3)  What  types  of  inputs  to  the  problem  do  you  have?  How  do  you  recognize  the 
problem?  Do  you: 

* see  something 

- hear  something 

- talk  to  someone 

- feel  something 

- read  something  - book,  test  equipment,  output  computer  display 

- detect  it  based  on  past  data 

**A4)  Does  solving  the  problem  require  a lot  of  data?  If  so,  about  what  and  how  do  you 

acquire  the  data?  Do  you  try  to  group  data  together  into  categories  that  have 
some  unifying  principle? 

A5)  Is  the  data  you  work  with  primarily  numerical  or  is  it  more  qualitative? 

A6)  What  assumptions  can  be  made  that  would  simplify  the  problem  solving  task? 
What  is  the  effect  when  these  assumptions  are  removed? 

A7)  What  could  happen  to  complicate  the  task? 

A8)  Are  there  manuals  or  desk  procedures  that  document/direct  what  you  do? 


A9) 

**A10) 

All) 

A12) 

**A13) 

A14) 

**A15) 

**A16) 

A17) 


When  you  are  defining/ identifying  the  problem,  do  you  spend  a lot  of  time  looking 
up  information  or  thinking  about  different  aspects  of  the  problem  or  do  you 
perform  the  task  mostly  automatically?  What  types  of  interactions,  if  any,  are 
required  in  terms  of  documents,  data  systems,  or  other  people  when  performing 
problem  definition? 

What  types  of  information  come  to  mind  when  you  do  the  task  in  the  sense 
of  inputs  into  the  problem? 

- facts 

- principles/rules/laws 

- procedures 

- similar  problems/analogies 

- past  problems 

Do  you  interact  with  other  people  while  you’re  defining  the  problem?  If  so, 
how?  Do  you: 

- advise 

- answer 

- direct 

- indicate 

- inform 

- instruct 

- request 

- transmit 

- supervise 

Do  you  think  body  language  such  as  gestures  and  facial  expressions  are 
important  in  communicating  at  this  time? 

Is  there  a lot  of  uncertainty  in  the  problem  --  maybe  even  to  the  extent  that 
it’s  hard  to  know  if  there  is  a problem?  How  do  you  go  about 
identifying/defining  the  problem? 

Do  the  problem  characteristics  change  frequently,  maybe  even  while  you’re 
solving  it?  If  so,  what  changes  --  data,  rules  by  which  you  solve  the  problem? 
What  drives  the  change  - e.g.,  time  intervals? 

Do  you  have  to  solve  the  problem  quickly  or  are  there  other  constraints  on  the 
problem?  e.g.,  Is  time  a factor  - what  is  "real-time"  in  this  situation  — seconds, 
minutes,  hours?  How  flexible  are  any  constraints  on  the  problem? 

Do  you  use  solutions  to  past  problems  to  help  you  solve  the  current  problem? 
How  similar  do  they  tend  to  be  and  how  do  you  modify  them? 

How  do  you  decide  what  past  problem  to  use?  Do  you  use  a specific  problem 
or  some  general  version? 


A 18) 

A19) 

A20) 


**A21) 


Do  you  try  to  draw  parallels  to  similar  but  not  identical  problems,  see  how 
those  problems  were  solved,  and  use  that  information  in  the  current  problem? 

What  equipment  do  you  use  while  defining  the  problem? 

How  many  alternative  answers  are  there  for  a problem?  Do  you  have  to  check 
for  many  as  you  solve  it?  Is  one  answer  obviously  better  than  another?  How 
can  you  tell?  Must  you  find  the  best  answer  or  is  a certain  level  of  answer 
good  enough? 

How  would  you  describe  the  output  of  your  task?  Do  you  generate  a 
document,  a decision,  a recommendation,  an  action,  a piece  of  equipment,  an 
interaction  with  someone,  etc.? 


A 22)  Can  you  list  all  possible  solutions  to  your  problem  at  the  start?  If  so, 

approximately  how  many  are  there? 


DEVISING  THE  PLAN 

**B23)  When  you  think  about  a problem  do  you: 

a)  visualize  it  in  your  mind 

b)  formalize  it  in  a set  of  equations  and  work  it  out  mathematically 

c)  explain  it  and  solve  it  in  English 

B24)  Do  you  calculate  or  look  at  statistics  to  help  you  solve  the  problem? 

B25)  Do  you  order  elements  of  the  problem  in  time  or  do  you  have  to  think  about 

a time  element  during  the  solution?  (i.e.,  such  as  data  or  input  changing  over 
time). 

B26)  Do  you  generate  a model  to  help  you  solve  the  problem,  such  as  a 

mathematical  model,  a physical  model,  or  some  other  symbolic  representation 
of  the  problem?  Do  you  "simulate"  the  behavior  of  a device  in  your  head. 

B27)  When  you  plan  the  job,  do  you  spend  a lot  of  time  looking  up  information  or 

thinking  about  different  aspects  of  the  problem  or  do  you  perform  the  task 
mostly  automatically?  What  types  of  interactions  are  required  in  terms  of 
documents,  data  systems,  or  other  people? 

B28)  Do  you  break  the  problem  into  parts?  Are  there  sequential  tasks  you  perform 

or  logically  distinct  subcomponents  of  the  problem?  If  so,  what  are  they? 

B29)  When  you  plan  the  task,  do  you  have  to  do  one  thing  after  another,  such  that 

one  set  of  conditions  causes  you  to  respond  with  a whole  series  of  actions?  Do 
you  wait  after  each  one  to  see  the  results  before  you  go  any  further? 


B30) 


Do  you  interact  with  other  people  while  planning  the  task?  If  so,  how?  Do 
you: 


- advise 

- answer 

- direct 

- indicate 

- inform 

- instruct 

- request 

- transmit 

- supervise 

- body  language 

B31)  What  type  of  formal  education  do  you  have?  Both  in  general  as  well  as  specific 

to  your  job.  What  role  does  experience  play?  Do  you  have  an 
apprenticeship?  How  do  you  become  qualified  for  what  you  do? 

B32)  Do  you  think  personal  factors  such  as  age,  gender,  height,  or  weight  influence 

yours  or  others  ability  to  do  the  task? 

B33)  Do  you  think  you  have  to  have  a certain  attitude  or  personality  to  work  well 

on  this  job? 

B34)  Do  you  generate  solutions  and  then  test  them  to  see  if  they  work? 

B35)  Do  you  think  of  past  solutions  and  then  modify  them  to  suit  the  current 

problem? 

B36)  Do  you  look  at  data  and  find  the  problem  or  a method,  then  look  at  more 

data,  etc.  Or  do  you  have  a goal  and  collect  data  to  meet  that  goal? 

B37)  When  you  solve  the  problem  do  you  have  a goal  and  take  incremental  steps 

in  an  attempt  to  step  by  step  get  to  that  goal? 

B38)  Are  there  set  procedures  or  formulas  you  use  for  your  task? 

B39)  Are  the  procedures  you  use  to  solve  the  problem  known? 

B40)  Do  you  gather  data  and  then  present  it  by  writing  about  it  or  discussing  it  with 

others? 

**B41)  Do  you  break  your  problem  into  sub-parts  and  then  solve  those?  After  you 
solve  the  sub-parts  do  you  have  to  reassemble  them  into  an  overall  solution? 

B42)  Do  you  ever  go  back  and  return  to  the  specifications  or  original  definition  of 

the  problem? 


B43)  Do  you  generate  solutions  that  explain  part  of  the  problem,  and  then 

determine  what  part  is  solved  and  try  to  generate  a solution  to  solve  any 
remaining  portions  of  the  problem? 

B44)  During  your  task  do  you  have  to: 

- calculate 

- code 

- computerize 

- interpolate 

- itemize 

- tabulate 

- translate 

- estimate 

- compute 

B45)  Do  you  choose  between  alternatives?  Do  you  compare  alternatives? 

B46)  Do  you  integrate  portions  of  the  problem  in  the  final  solution? 

B47)  Do  you  have  to  do  a lot  of  planning?  Plan  the  task  --  or  create  a plan? 

B48)  Do  you  categorize  parts  of  the  problem  together?  Or  categorize  data  in  order 

to  solve  the  problem? 

B49)  Are  there  general  principles  that  you  know  to  be  true  in  your  field?  Do  you 

use  those  to  help  determine  what’s  going  on  in  a specific  situation? 

**B50)  If  you  make  a wrong  choice,  is  it  easy  to  undo  and  if  it  is,  how  do  you  undo 
it? 

B51)  What  equipment  do  you  use  when  planning  the  problem  solution? 

CARRYING  OUT  THE  PROBLEM 

**C52)  What  are  the  major  steps  you  perform? 

C53)  Where  is  your  problem  solving  generally  performed?  Describe  your  work 

environment.  Do  you  work  at  a desk  with  paper  and  pencil,  at  a console  or 
in  front  of  a piece  of  equipment,  walking  around  and  talking  to  people,  or 
flying  an  aircraft?  Is  there: 


- acceleration 

- confinement 

- isolation 

- contaminants  - what  are  they? 

- electricity 

- lighting 

- magnetism 

- noise 

C54)  How  demanding  is  the  task  physically  and  psychologically? 

- fatigue 

- mental  strain 
* stress 

- physical  strain 

- precision 

- amount  of  sleep 

- work  schedule 

- have  to  pay  attention  all  the  time 

C55)  Do  you  have  to  pay  close  attention  to  the  task  for  long  periods  of  time? 

C56)  What  types  of  physical  movements  do  you  make  when  you  do  the  job? 

(we  will  assess  motor  processes  and  perceptual  requirements  from  the 
demonstration) 

C57)  Do  you  interact  with  other  people  during  the  actual  execution  of  the  task? 

How?  Do  you: 

- advise 

- answer 

- direct 

- indicate 

- inform 

- instruct 

- request 

- transmit 

- supervise 

- body  language 

C58)  When  you  do  the  job,  do  you  spend  a lot  of  time  looking  up  information  or 

thinking  about  different  aspects  of  the  problem  or  do  you  perform  the  task 
mostly  automatically?  What  types  of  interactions  are  required  in  terms  of 
documents,  data  systems,  or  other  people  during  task  execution? 

C59)  Do  you  monitor  some  set  of  indicators  as  the  task  progresses? 

C60)  Do  you  have  to  interpret  information  you  receive  maybe  forming  new 

indicators? 


C61)  What  type  of  equipment  is  involved  in  actually  performing  the  task? 

C62)  When  you  perform  the  job,  do  you  have  to  do  one  thing  after  another,  such 

that  one  set  of  conditions  causes  you  to  respond  with  a whole  series  of 
actions?  Do  you  wait  after  each  one  to  see  the  results  before  you  go  anv 
further?  6 7 

**C63)  What  is  the  hardest/easiest  thing  you  do?  What  is  the  most  important 
D)  LOOKING  BACK 


**D64)  How  many  alternative  answers  do  you  usually  come  up  with?  Do  you  have  to 
check  for  many  as  you  solve  it?  Is  one  answer  obviously  better  than  another? 
How  can  you  tell?  Must  you  find  the  best  answer  or  is  a certain  level  of 
answer  good  enough? 

D65)  Do  you  interact  with  other  people  in  verifying  the  task?  How?  Do  you: 

- advise 

- answer 

- direct 

- indicate 

- inform 

- instruct 

- request 

- transmit 

- supervise 

- body  language 

D66)  When  you  evaluate  your  solution,  do  you  spend  a lot  of  time  looking  up 

information  or  thinking  about  different  aspects  of  the  problem?  w ' 

D67)  Can  the  quality  of  a solution  be  characterized  as  better  or  worse  rather  than 

acceptable/not  acceptable? 
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REVISED  FIRST  INTERVIEW  QUESTIONNAIRE 


Appendix  E 

Revised  First  Interview  Questions 


Initial  Discussion: 

There  are  no  right  answers  or  good  answers  to  these  questions.  Some  of  the  questions  may 
be  phrased  so  that  you  believe  one  or  another  response  is  appropriate  or  somehow  better  than 
another.  For  example,  it  may  sound  good  to  say  you  plan  up  front  all  your  actions  in  solving  a 
problem  or  doing  the  job  even  if  some  of  those  actions  occur  somewhat  automatically.  Please  do 
not  fall  into  this  bias.  For  this  example,  it  is  good  to  plan  but  it  is  also  good  not  to  plan  too  much 
and  to  have  some  actions  that  occur  automatically.  We  want  to  assess  all  aspects  of  your  job  with 
as  much  fidelity  as  possible.  What  results  from  these  interviews  will  help  us  to  better  understand 
the  unique  aspects  of  your  work  and  how  we  could  learn  more  about  it  in  the  future  so  we  can 
design  computer  systems  that  help  you  perform  your  job  more  easily. 

Also,  we  want  to  assure  you  that  under  no  circumstances  will  your  answers  go  to  your 
supervisor  or  anyone  else  outside  of  this  room.  You  are  responding  to  this  questionnaire 
completely  anonymously.  Because  of  this,  we  want  you  to  provide  the  operational  answers  to  these 
questions,  not  the  official  ones.  We  want  to  know  what  you  really  do  on  your  job. 


A)  UNDERSTANDING  THE  PROBLEM 

Al)  Tell  us  a little  bit  about  your  field,  yourself,  and  how  you  got  into  this  field. 

A2)  What  type  of  formal  education  do  you  have,  both  in  general  as  well  as  specific  to 

your  job?  What  role  does  experience  play?  Did  you  have  an  apprenticeship? 
How  do  you  become  qualified  for  what  you  do? 

A3)  About  how  long  does  it  take  to  perform  the  task  (start  to  finish)  --  is  there  a start 
and  a finish,  how  well  defined  are  they  and  what  defines  them? 

A4)  What  types  of  inputs  to  the  problem  do  you  have?  How  do  you  recognize  the 
problem?  Do  you: 

- see  something 

- hear  something 

- talk  to  someone 

- feel  something 

- read  something  - book,  test  equipment,  output  computer  display 

- detect  it  based  on  past  data 

A5)  Does  solving  the  problem  require  a lot  of  data?  If  so,  about  what  and  how  do  you 
acquire  the  data?  Do  you  try  to  group  data  together  into  categories  that  have 
some  unifying  principle?  Is  the  data  you  work  with  primarily  numerical  or  is  it 
more  qualitative? 

A6)  Are  there  manuals  or  desk  procedures  that  document/direct  what  you  do? 


A7)  What  types  of  information  come  to  mind  when  you  do  the  task  in  the  sense  of 
inputs  into  the  problem? 

- facts 

- principles/rules/laws 

- procedures 

- similar  problems/analogies 

- past  problems 

A8)  Is  there  a lot  of  uncertainty  in  the  problem  - maybe  even  to  the  extent  that  it’s 
9hard  to  know  if  there  is  a problem?  How  do  you  go  about  identifying/defining 
the  problem? 

A9)  Do  you  have  to  solve  the  problem  quickly  or  are  there  other  constraints  on  the 
problem?  e.g.,  Is  time  a factor  - what  is  "real-time"  in  this  situation  - seconds, 
minutes,  hours?  How  flexible  are  any  constraints  on  the  problem? 

A 10)  Do  you  use  solutions  to  past  problems  to  help  you  solve  the  current  problem? 

How  similar  do  they  tend  to  be  and  how  do  you  modify  them?  How  do  you  decide 
what  past  problem  to  use?  Do  you  use  a specific  problem  or  some  general 
version? 

All)  Can  you  list  all  possible  solutions  to  your  problem  at  the  start?  If  so, 
approximately  how  many  are  there? 


DEVISING  THE  PLAN 

B12)  When  you  think  about  a problem  do  you: 

a)  visualize  it  in  your  mind 

b)  formalize  it  in  a set  of  equations  and  work  it  out  mathematically 

c)  explain  it  and  solve  it  in  English 

B13)  Do  you  order  elements  of  the  problem  in  time  or  do  you  have  to  think  about  a 
time  element  during  the  solution?  (i.e.,  such  as  data  or  input  changing  over  time). 
Do  you  have  to  do  a lot  of  planning?  Plan  the  task  --  or  create  a plan? 

B14)  Do  you  generate  a model  to  help  you  solve  the  problem,  such  as  a mathematical 
model,  a physical  model,  or  some  other  symbolic  representation  of  the  problem? 
Do  you  "simulate"  the  behavior  of  a device  in  your  head? 

B 15)  Do  you  break  the  problem  into  sub-parts?  Are  there  sequential  tasks  you  perform 
or  logically  distinct  subcomponents  of  the  problem?  If  so,  what  are  they?  After 
you  solve  the  sub-parts  do  you  have  to  reassemble  them  into  an  overall  solution? 

B16)  Do  you  generate  solutions  and  then  test  them  to  see  if  they  work  or  do  you  think 
of  past  solutions  and  then  modify  them  to  suit  the  current  problem? 


B17)  When  you  solve  the  problem  do  you  have  a goal  and  take  incremental  steps  in  an 
attempt  to  step  by  step  get  to  that  goal? 

B18)  Are  there  set  procedures  or  formulas  you  use  for  your  task?  Are  there  general 
principles  that  you  know  to  be  true  in  your  field?  Do  you  use  those  to  help 
determine  what’s  going  on  in  a specific  situation? 

B19)  Do  you  ever  go  back  and  return  to  the  specifications  or  original  definition  of  the 
problem? 

B20)  Do  you  generate  solutions  that  explain  part  of  the  problem,  and  then  determine 
what  part  is  solved  and  try  to  generate  a solution  to  solve  any  remaining  portions 
of  the  problem? 

B21)  Do  you  choose  between  alternatives?  Do  you  compare  alternatives? 


CARRYING  OUT  THE  PROBLEM 

C22)  What  are  the  major  steps  you  perform? 

C23)  Where  is  your  problem  solving  generally  performed?  Describe  your  work 
environment.  Do  you  work  at  a desk  with  paper  and  pencil,  at  a console  or  in 
front  of  a piece  of  equipment,  walking  around  and  talking  to  people,  or  flying  an 
aircraft? 

C24)  How  demanding  is  the  task  physically  and  psychologically? 

C25)  Do  you  have  to  pay  close  attention  to  the  task  for  long  periods  of  time? 

C26)  What  types  of  physical  movements  do  you  make  when  you  do  the  job? 

C27)  Do  you  interact  with  other  people  during  the  actual  execution  of  the  task?  How? 

C28)  Do  you  gather  data  and  then  present  it  by  writing  about  it  or  discussing  it  with 

others? 

C29)  Do  you  monitor  some  set  of  indicators  as  the  task  progresses? 

C30)  Do  you  have  to  interpret  information  you  receive  maybe  forming  new  indicators? 

C31)  Do  you  calculate  or  look  at  statistics  to  help  you  solve  the  problem? 

C32)  What  type  of  equipment  is  involved  in  actually  performing  the  task? 

C33)  When  you  perform  the  job,  do  you  have  to  do  one  thing  after  another,  such  that 

one  set  of  conditions  causes  you  to  respond  with  a whole  series  of  actions?  Do 
you  wait  after  each  one  to  see  the  results  before  you  go  any  further? 


C34)  Do  you  think  personal  factors  such  as  age,  gender,  height,  or  weight  influence 
yours  or  others  ability  to  do  the  task?  Do  you  think  you  have  to  have  a certain 
attitude  or  personality  to  work  well  on  this  job? 

C35)  What  could  happen  to  complicate  the  task? 

C36)  If  you  make  a wrong  choice,  is  it  easy  to  undo  and  if  it  is,  how  do  you  undo  it? 

LOOKING  BACK 

D37)  How  would  you  describe  the  output  of  your  task?  Do  you  generate  a document, 
a decision,  a recommendation,  an  action,  a piece  of  equipment,  an  interaction  with 
someone,  etc.? 

D38)  How  many  alternative  answers  do  you  usually  come  up  with?  Do  you  have  to 
check  for  many  as  you  solve  it?  Is  one  answer  obviously  better  than  another? 
How  can  you  tell?  Must  you  find  the  best  answer  or  is  a certain  level  of  answer 
good  enough? 

D39)  How  do  you  evaluate  your  solution?  Can  the  quality  of  a solution  be  characterized 
as  better  or  worse  rather  than  acceptable/not  acceptable? 

D40)  What  is  the  hardest/easiest  thing  you  do?  What  is  the  most  important? 
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4 1 

4 1 

2 1 

4 1 

3 1 

01 

31 

o 1 

4 I 

4 I 

4 I 

4 

medical  diag  | 

4 1 

4 1 

2 j 

4 1 

2 1 

o| 

2 1 

o 1 

1 j 

0 j 

2 1 

4 

accounting  | 

0 1 

0| 

0 I 

1 j 

3 1 

4 1 

0| 

3 | 

0 1 

0 j 

o 1 

2 

protocol  des  | 

Q 1 

0| 

2 I 

4 1 

3 1 

3 1 

21 

3 j 

4 I 

1 j 

0 1 

4 

( 


A I VAX  $ DU A 1 : [ A I . KATAX ] RECALL_PERCE P . LIS ; 1 1 3- JUN-199  I Q9 : 14 
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AI  VAX  $ OUAi  : { A I . KATAX  J PHYSJNVIR  . LIS  ; 1 


i J - J UN- 1 9 9 1 Q9  ; 14 


t a s k_name  | acce X & r a 1 1 on  J c on f 1 nement  | isolation  j contaminants (electricity | lighting (magnetism (noise 


drair  gen  | 

0| 

0 1 

0| 

0| 

0 1 

o 1 

0 I 

0 

program  mgmt  j 

o| 

o 1 

0| 

Q 1 

0 j 

Q j 

0 1 

0 

weather  | 

0 1 

o 1 

0| 

0| 

0 1 

a | 

0 1 

0 

console  ops  | 

0| 

1 1 

0| 

0| 

0 1 

1 1 

0 1 

4 

air  traffic  | 

0| 

3 I 

0| 

0| 

o 1 

2 1 

0 1 

3 

pilot  training  | 

3| 

4 1 

0| 

0 1 

0 1 

0 1 

0 1 

4 

equip  diag  | 

0 1 

0 1 

0| 

2| 

3 j 

0 1 

0 1 

4 

language  s rain  j 

0| 

0 j 

0| 

0| 

0 j 

0 1 

0 1 

0 

for»Jill_out  j 

0| 

0 j 

0| 

0| 

0 j 

0 1 

0 1 

0 

sw  de sign  | 

0 1 

0 1 

0 1 

°l 

0 1 

0 j 

o 1 

0 

cargo  loading  | 

11 

2 1 

0 1 

0| 

0 I 

o i 

0 j 

3 

leadership  | 

0| 

0 1 

0| 

0| 

0 1 

0 ] 

0 1 

0 

surgery  | 

0| 

2 1 

0| 

11 

o i 

2 1 

a 1 

0 

Radical  dug  j 

0| 

0 j 

0| 

0| 

0 j 

0 j 

0 1 

0 

accounting  | 

0| 

o 1 

0| 

0| 

0 1 

0 1 

0 1 

0 

p rotoco 1 de  s | 

0| 

o 1 

0| 

0 1 

o 1 

0 j 

0 j 

0 

AI VAX $ DUA1  : [ AI  . KATAX ] PS YCH  EN V I R . L I S ; 1 


13-JUN-1991  09:15 


task  name  | fatigue |aent_ 

strain | stress \ phy s 

strain (preciseness |cog 

a 1 1 an  t j response 

cha i ng  j a 1 1 an  t 

ion  span | 

sleep  j 

schedule | boredon 

drair  gen  j 

0| 

0| 

0| 

0| 

0| 

0 | 

0 t 

0 1 

0 1 

0 j 0 

prograi  mg rot  | 

°l 

0| 

0| 

0| 

0| 

0 1 

0 1 

0 1 

0 j 

0 j 0 

weather  j 

°l 

0| 

0| 

0| 

0| 

o 1 

0 1 

0 j 

0 1 

0 | 0 

console  ops  | 

M 

2| 

3| 

11 

M 

4 j 

1 ( 

3 j 

0 I 

2 j 2 

air  traffic  j 

4 1 

4| 

4I 

0| 

0| 

4 1 

3 1 

4 I 

4 1 

4 j 0 

pilot  training  j 

31 

j| 

3 1 

11 

3 1 

4 1 

1 1 

3 | 

0 1 

3 j 0 

equip  diag  | 

0| 

0| 

0| 

3 I 

0| 

0 1 

0 i 

1 j 

0 j 

2 j 3 

language  train  j 

0| 

°l 

0| 

0| 

0| 

0 j 

0 j 

0 j 

0 j 

0 j 0 

Core  fill  out  | 

0| 

0| 

0| 

0| 

0| 

0 I 

0 I 

0 j 

0 j 

0 1 0 

sw  design  | 

0| 

0| 

0| 

0| 

0| 

0 j 

0 1 

0 j 

0 j 

0 j 0 

cargo  loading  | 

0| 

0| 

0| 

11 

0| 

0 I 

0 j 

o 1 

0 1 

0 j 2 

leadership  | 

0| 

0| 

0| 

0| 

0| 

o 1 

0 j 

0 j 

o j 

0 j 0 

surgery  | 

3| 

3 1 

4 1 

3 1 

4 1 

4 j 

2 I 

4 1 

2 j 

4 j 1 

Medical  diag  | 

11 

2| 

3 1 

0| 

0| 

0 j 

0 1 

0 j 

1 | 

2 j 1 

accounting  | 

0| 

0| 

0| 

0| 

4 1 

0 I 

0 1 

0 1 

0 I 

0 j 0 

protocol  des  | 

0| 

0| 

0| 

0| 

0| 

0 1 

o I 

0 1 

o 1 

0 j 0 

AIVAX$DUAl : ( AI . KATAX  j PERSONAL  PH YS . LIS ; 1 


1 3 - J UN - 1 9 9 1 0 9:1*? 


task_na«e  J genera  l__health  | age  (gender  (height  | weight 


drair  gen  | 

0| 

0| 

0| 

0| 

0 

program  ngmt  j 

0| 

0| 

0| 

0| 

0 

weather  j 

°l 

0| 

°l 

0| 

0 

console  ops  | 

<1 

H 

°l 

0| 

2 

airtraffic  j 

* 1 

0| 

0| 

0| 

0 

pilottraining  j 

4| 

3| 

2| 

11 

3 

•quip  diag  | 

0| 

0| 

0| 

0| 

Q 

language  train  j 

0| 

0| 

0| 

0| 

0 

f ora_fi ll_out  j 

°l 

0| 

0| 

0| 

0 

sw  de sign  | 

0| 

0| 

0| 

0| 

0 

cargo  loading  | 

2| 

0| 

01 

0| 

0 

leadership  j 

0| 

°l 

0| 

0| 

0 

surgery  j 

0| 

o| 

0| 

0| 

0 

Medical  diag  j 

0| 

0| 

0| 

0| 

0 

accounting  j 

0| 

o| 

0| 

0| 

0 

protocol  des  | 

0 1 

0| 

0| 

0| 

0 

i 


C 


AI VAX  $ DUAl  : { AI  . KATAX ] P E R S ON AL  PS Y CH . LIS ; 1 


13-JUN-1991  09  : 16 


task  name  | edu_t r a i n_e  x pe r t | intro_accuracy | p cob_of ^success | articulation) kr_hp 


dr  a i r__gen 
prog  r aa^agnt 

i 

i 

2| 

1| 

o 1 
o 1 

0 1 

1 j 
1 1 

4 I 
0 1 
0 1 

1 

1 

1 

weather 

i 

2| 

0 | 

J 

console  ops 

i 

ji 

0 1 

s \ 

c I 

air  traffic 

i 

31 

0 1 

3 j 

3 | 

i 

pilottraining 
equip  diag 
language  train 

1 

1 

1 

3| 

M 

2| 

3 1 
a 1 

4 1 

3 | 
0 1 
2 1 

4 | 
0 1 
4 j 

n i 

3 

0 

1 

fora  fill  out 

1 

2 1 

0 | 

0 | 

u 1 

sw  design 

1 

3| 

0 1 

0 1 

3 j 

i i 

1 

i 

ca  r go  loading 

1 

2 1 

0 I 

0 | 

1 | 
A 1 

i 

i 

leade  r sh ip 

1 

M 

4 j 

3 | 

H \ 

l 

surgery 

1 

3| 

0 1 

4 j 

1 1 

3 

l 

aedical  diag 

1 

3| 

3 1 

4 | 

J 1 

account i ng 

1 

3 1 

0 1 

0 j 

0 | 

1 

1 

protocol  de  s 

1 

3 1 

0 j 

4 | 

J 1 

A I VAX  $ DUA1 : ( AI . KATAX 1 EQUIP  TOOLS . LIS ; 1 


13-JUN-1991  09:16 


t a s k name  | computer j vehicles | weapon  systems J instruments |notation|testequip 


drair_gen  | 

4 | 

0| 

0| 

1 | 

0 

program  mgmt  | 

2 1 

0| 

0| 

0| 

0 1 

0 

weather  j 

1 | 

0| 

0| 

M 

4 j 

0 

console  ops  j 

4 1 

0| 

0| 

3| 

2 | 

1 

air  traffic  j 

0 1 

01 

0| 

<1 

4 1 

0 

pilot  training  j 

0 j 

0| 

<1 

2 1 

Q 

equip_diag  | 

3 1 

0| 

0| 

01 

4 | 

0 

language  t ra in  | 

2 | 

0| 

0| 

0| 

0 j 

0 

form^fill  out  | 

4 1 

o| 

0| 

0| 

3 1 

0 

sw  design  | 

4 I 

0| 

0| 

0 1 

4 1 

0 

cargo  loading  j 

0 1 

2| 

0 1 

0| 

2 1 

0 

Leadership  | 

2 1 

0| 

0| 

0| 

2 j 

0 

surgery  | 

0 j 

0| 

0| 

<1 

4 1 

3 

medical  diag  | 

0 j 

0| 

0| 

2 1 

4 1 

2 

accounting  | 

4 1 

01 

0| 

0| 

3 1 

0 

protocol  des  j 

2 j 

o| 

0| 

*\ 

0 1 

1 

I 


APPENDIX  H 


TASK  CHARACTERISTICS  RATING  MATRIX  AFTER  SECOND  INTERVIEW 


AI V AX  $ OUA1  : { A I . KATAX ] RBASOHING  TECHS 2 . LIS  ; 9 


12-JUL-1991  15:21 
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task  name  | sta t 

is ticai J 

spatial  | 

t eapo ral| 

analogical (case 

based | model 

based | 

Mathematical | 

deductive  | 

induct i ve 

drair^gen  | 

ii 

0 1 

M 

t>l 

0 1 

o 1 

3 1 

4 1 

0 

prograi  mgmt  | 

0| 

0 1 

3| 

0 1 

3 | 

0 | 

1 | 

0 1 

2 

weather  | 

21 

3 | 

4 I 

0| 

2 1 

4 I 

2 I 

3 I 

0 

console_ops  | 

0| 

2 1 

4 I 

0| 

2 | 

3 I 

0 1 

2 I 

0 

air  traffic  | 

0| 

4 ) 

4 | 

o| 

0 i 

1 | 

0 1 

0 1 

1 

pilot  training  | 

0| 

4 1 

3 I 

°l 

1 | 

3 ) 

0 | 

0 | 

0 

equipdiag  | 

2| 

1 | 

M 

0| 

0 | 

2 | 

o 1 

3 | 

0 

language^  rain  | 

*1 

0 | 

ji 

3 I 

2 I 

Q | 

0 I 

2 1 

2 

f o rm_f i 1 l_out  | 

<>| 

0 1 

0| 

0| 

3 | 

0 1 

0 1 

1 | 

0 

$w  design  | 

0\ 

0 I 

4 1 

0| 

3 { 

2 1 

2 I 

0 1 

0 

ca rgo_loading  | 

0\ 

1 1 

2 1 

0| 

1 | 

2 1 

4 I 

1 f 

0 

leadership  J 

0| 

0 | 

3| 

0| 

4 | 

1 | 

o 1 

0 1 

0 

surgery  | 

0| 

4 1 

2| 

°l 

1 | 

4 1 

0 | 

1 | 

0 

m*dical_diag  | 

H 

Oj 

4 I 

0| 

21 

31 

0 | 

4 I 

0 

accounting  | 

0| 

0 1 

2| 

0| 

1 | 

0 | 

4 ( 

3 | 

0 

protocol  des  | 

«l 

0 I 

J| 

0| 

2 | 

0 | 

o 1 

0 1 

0 

1 2- JUL-i 9 9 i i 5 : 1 9 
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AI V AX  $ DUAl  : ( AX  . KATAX  | PROflJO  LVJECHS  1 2 . LIS  ; 


task  name  | naans^ends j sub 

goals  | gen 

andtest  | goal_ 

vs  data | cove  r 

and_di f f | p t op_ 

dnd  r e £ | acq_ 

and  pres 

drairgen  1 

0| 

3 1 

0| 

D| 

0| 

4 

prograiagit  | 

11 

3 t 

M 

G| 

0| 

It 

3 

weather  | 

0| 

3 1 

2 1 

GiD  | 

3 | 

o 1 

4 

console^ops  j 

<M 

2 1 

0| 

D| 

3| 

0| 

1 

air  traffic  | 

4| 

4 1 

0| 

GWD  j 

G| 

0| 

2 

p i lot_t r a in ing  | 

3| 

3 1 

<M 

<=! 

0| 

2 

•quip  diag  | 

0| 

2 1 

4 I 

D| 

11 

0| 

2 

language_tnin  | 

3| 

4 1 

2 1 

2 I 

°l 

0 

fora  fill  out  | 

21 

3| 

0| 

<51 

0| 

2| 

4 

sw  design  | 

4| 

4 1 

4 1 

G| 

01 

3 1 

4 

cargo  loading  | 

21 

2 1 

M 

»l 

01 

®l 

1 

leadership  | 

3| 

*\ 

3 1 

G| 

11 

11 

4 

surgery  I 

*\ 

4| 

0| 

G| 

1 1 

0| 

0 

■edicaldiag  | 

0| 

0| 

3| 

D| 

4 1 

0| 

2 

accounting  I 

2| 

4 1 

3 1 

G| 

0| 

3 

protocoidas  1 

41 

4 1 

3 1 

G| 

®l 

3 1 

3 

12-JUL-i99i  12:22 
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AX  VAX  $ DUA1  : ( AI . KATAX ] PROB_SOLV_TECHS 2 2 , LIS; 2 


task  narae  j to rni_proc_alg (decompose | recoabine (generalize! specialize | ret_to_def 


drai r gen  | 

2 1 

2| 

2 1 

2 1 

0 1 

0 

program  agat  | 

0 1 

2| 

a j 

1| 

1 | 

2 

weather  j 

2 j 

4 1 

3 j 

<>l 

3 I 

3 

console  ops  | 

1 | 

2| 

o 1 

0| 

2 I 

3 

air  traffic  | 

3 | 

4| 

o 1 

0| 

3 I 

0 

pilot  training  | 

3 | 

3| 

3 | 

Oj 

0 j 

0 

equip  diag  | 

4 1 

3| 

1 | 

°l 

3 1 

4 

language^  rain  j 

3 1 

4| 

4 j 

31 

2 I 

3 

fora  fill__out  j 

4 j 

3| 

0 1 

01 

2 j 

2 

sw  design  j 

1 | 

4| 

4 I 

»l 

2I  , 

1 

cargo  Loading  | 

4 j 

2 1 

2 j 

0| 

2 | ? 

2 

leadership  | 

0 1 

«l 

2 j 

M 

2 j 

0 

surgery  | 

3 j 

4 1 

0 1 

«l 

3 j 

2 

aedicaldiay  | 

1 j 

»l 

0 1 

•1 

2 | 

2 

accounting  | 

4 j 

4 1 

2 | 

0| 

2 1 

2 

protocol  des  | 

o | 

H 

1 j 

0| 

3 1 

2 

C 


mm  mi 


( (' 

1 2- JUL-1 99  1 12:40 


an  | i 

nstru mentation |mm_ 

interface  | 

databases | 

historical jpropadeut 

ICS 

1 | 

o t 

°i 

4 1 

4 1 

3 

3 | 

o 1 

01 

ii 

2 | 

1 

4 1 

2 I 

<1 

2| 

3| 

1 

2 1 

0 

4 1 

0 1 

0 1 

1 

2 1 

4 

4 I 

1 1 

0 1 

0 

2 | 

4 1 

4 I 

0 1 

0| 

1 

4 | 

3 | 

o\ 

0| 

3| 

2 

4 1 

o 

0\ 

0| 

1| 

1 

4 1 

o 

0| 

3| 

2 1 

4 

4 | 

0 

0 1 

0 1 

0 1 

4 

3 1 

0 1 

°l 

0| 

°| 

1 

4 1 

0 1 

0| 

0| 

0| 

0 

0 1 

4 

0 1 

0| 

0| 

4 

4 1 

4 

0 1 

0| 

41 

1 

4 | 

0 

2 1 

41 

3| 

2 

4 1 

0 1 

0| 

01 

1 1 

3 

AI VAX  $ DUAl  : [ A I . KATAX ] TASKCOMPLEX 2 , LIS  ; 3 
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task  name  |branching|dynaeisn|constraints (uncertainty!  h i gh_low_tech | quant_qua 1 


drair  gen  | 

0| 

0| 

0| 

0 1 

2 1 

2 

prograi  igat  | 

0| 

0| 

2 1 

1 j 

1 | 

3 

weather  I 

4| 

3 1 

2 1 

4 1 

3 | 

2 

console  ops  } 

2| 

4| 

< 1 

3 | 

3 1 

2 

air  traffic  j 

31 

4I 

1 j 

1 1 

2 

pilot  training  j 

<1 

4 1 

2 1 

0 1 

2 | 

2 

•quip  diag  | 

2| 

0| 

11 

1 | 

3 | 

2 

language_t r ain  | 

2| 

2| 

2 1 

1 | 

1 j 

3 

f o rn_f il l_out  j 

1| 

0| 

3 1 

1 | 

1 j 

3 

sw  design  | 

<1 

0| 

<>l 

1 j 

3 j 

2 

cargoloading  | 

2| 

0| 

2 1 

0 j 

1 j 

1 

leadership  j 

3| 

0| 

1| 

2 | 

1 j 

3 

surgery  j 

3 1 

3| 

3 1 

1 j 

3 1 

3 

nedica l__di  ag  | 

«l 

2| 

2 1 

3 | 

3 1 

2 

accounting  | 

"I 

o| 

3 1 

0 j 

3 1 

1 

protocol  des  | 

<1 

0| 

2 1 

o 1 

3 j 

2 

12-JUL-1991  12:59 
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A I VAX  $ DU A 1 : [ AI  . KATAX  JM0T0R  PR0C2  . LIS ; 


| complex  cont  | compound  J reflex  j simple_disc  | fine  (gross  | repetitive 


drair  gen  | 

0| 

0| 

0| 

0| 

0| 

o 1 

0 

program  mgmt  | 

0| 

0| 

0| 

0| 

0| 

0 I 

0 

weather  | 

®i 

°l 

0| 

0 1 

o 1 

0 | 

conso le  ops  | 

0| 

0| 

0 1 

4| 

4 1 

0 j 

2 

air  traffic  j 

°l 

0| 

0| 

1 1 

1 1 

0 I 

0 

pilot  training  | 

0| 

0| 

11 

4 1 

1 1 

2 j 

3 

•quip  diag  | 

0| 

0| 

0| 

2 1 

2 1 

2 I 

1 

language  train  | 

0| 

0| 

0| 

o 1 

o 1 

0 | 

0 

form  fill__out  | 

0| 

0| 

01 

0| 

o 1 

0 j 

0 

sw  design  | 

0| 

0| 

0| 

G 1 

0| 

0 j 

0 

ca r go_loading  | 

0| 

0| 

0| 

0 1 

0 1 

0 | 

0 

leadership  | 

0| 

0| 

0| 

0| 

0 1 

0 | 

0 

surgery  t 

«l 

31 

0| 

4 1 

4 1 

1 I 

0 

medical  diag  j 

0| 

0| 

0| 

2 1 

2 1 

0 | 

0 

accounting  | 

0| 

0| 

0| 

0| 

0| 

o 1 

0 

protocol  des  | 

0| 

0| 

0| 

0 1 

o| 

0 | 

0 

AI VAX  $ DUAX  : l AI . KATAX ] INFOPROC2 . LIS; 2 


1 2 - JU L~ 19  91  13  : 09 


task  name  jcategorize  J calculate  J code  Jcoaputerize | interpolate | itemze | learn | tabulate  j translate 


drair  gen  | 

2 1 

4 1 

0 1 

0 I 

M 

o 1 

1 | 

2 | 

2 

prograa  agat  | 

1 j 

2 j 

0 I 

0 1 

0| 

2 j 

1 | 

o 1 

0 

weather  | 

4 1 

a 1 

0 j 

o 1 

4I 

1 j 

2 j 

1 j 

3 

console  ops  | 

3 1 

2 | 

Q j 

0 i 

3| 

0 j 

3 | 

o 1 

3 

air  traffic  j 

2 1 

2 1 

0 1 

0 1 

3| 

o 1 

0 j 

0 1 

1 

pilot  training  | 

1 | 

0 | 

0 1 

0 1 

2| 

2 1 

o 1 

Q j 

0 

equip  diag  j 

a j 

0 1 

0 j 

0 1 

Of 

2 t 

1 1 

o 1 

2 

language  ! r a in  | 

2 | 

0 1 

0 1 

0 1 

0| 

2 1 

0 1 

0 1 

4 

f ornf ill_out  j 

3 j 

1 1 

0 1 

o 1 

0| 

1 j 

1 j 

1 | 

4 

sw  design  | 

2 | 

1 1 

4 | 

4 1 

0| 

2 1 

1 1 

0 j 

4 

cargo  loading  | 

2 I 

4 I 

0 1 

0 1 

®l 

3 | 

1 1 

o 1 

0 

leadership  1 

1 j 

1 1 

o 1 

0 1 

0| 

4 j 

2 j 

o 1 

2 

surgery  | 

4 j 

0 I 

0 j 

o i 

0| 

3 | 

0 | 

0 j 

0 

Radical  diag  | 

4 j 

0 j 

0 1 

0 1 

0| 

1 j 

1 j 

o I 

3 

accounting  | 

4 1 

4 1 

3 1 

0 j 

0| 

4 1 

1 | 

4 j 

0 

protocol  des  | 

2 1 

0 1 

0 | 

0 1 

0| 

o 1 

1 | 

o 1 

4 
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task  na»e 


drai r^gen 

prograa_agnt 

weather 

consol e_ops 

a i r__t  r a f f ic 

pi lot_t raining 

•quip_diag 

languaga_train 

to r »_f i 1 l_out 

swedes i gn 

ca  r go_l oading 

leadership 

surgery 

■ edi c a I d l a g 

accounting 

protocol_des 


J facts 


principles]  procedures  | analogs  | ca  s e s_e  xanp  1 e s | percepspead  | search^recjnfo  | id_ob  jacte  vents  | scanjisplay 


1 

4 1 

0| 

0| 

°| 

3 I 

o 1 

0 1 

1 

3 | 

n 

x| 

o| 

3 1 

0 1 

0 1 

1 

4 1 

* 1 

21 

0| 

1 | 

0 1 

2 1 

1 

4 1 

«l 

4 1 

o| 

2 1 

4 1 

4 1 

1 

3 1 

21 

4 1 

o| 

Of 

2 1 

4 i 

1 

4 I 

«l 

4 1 

o| 

1 | 

2 1 

4 1 

1 

4 1 

M 

4 1 

o| 

2 | 

0 1 

0 1 

1 

4 I 

M 

X| 

2 1 

3.j 

o 1 

0 | 

1 

3 | 

0| 

M 

°| 

3 | 

0 1 

0 j 

1 

3 1 

<1 

o| 

0| 

2 1 

0 1 

o 1 

1 

4 1 

II 

2| 

0| 

0| 

0 1 

0 1 

1 

3 | 

2| 

2 1 

0| 

1 | 

0 1 

0 1 

1 

4 1 

<1 

4 1 

«:| 

xi 

0 j 

4 j 

1 

4 ! 

*1 

4 1 

0| 

3 1 

0 j 

2 1 

1 

2 | 

1| 

4 1 

x 1 

0 I 

1 j 

1 

2 1 

<1 

1| 

4 j 

0 1 

0 1 

0| 
0| 
4| 
4| 
31 
3 I 

3 I 
01 
0| 
<H 
0| 
0| 

4 I 
4 I 
M 
o| 


o 

0 

0 

4 

4 

4 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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task  name 


| acceleration (confinement | isolation | contaninants | electricity | lightinql magnet  is*  j noise 


d r a i r gen  | 

°i 

0 

progra*  agmt  ( 

0| 

0 

weather  | 

o| 

0 

console  ops  | 

0| 

0 

air  traffic  j 

0| 

1 

pilot  training  j 

3| 

4 

equip  diag  | 

0| 

0 

language_t ra in  j 

0| 

0 

f or»_f i 1 l_ou t j 

0| 

o 

s w de s i gn  | 

0| 

o 

ca  r go  loading  | 

0| 

0| 

leadership  j 

0| 

o 

surgery  | 

0| 

o 1 

*edical_diag  j 

0| 

o 

accounting  | 

0| 

o 

protocol  des  | 

0| 

0 

0| 

0| 

o 1 

0 

0| 

o| 

o 1 

a 

0| 

0| 

o 1 

0 

0| 

<M 

0 1 

1 

0| 

0| 

0 1 

3 

0| 

0| 

o 1 

0 

0| 

3| 

2 1 

Q 

01 

0| 

o I 

0 

0| 

0| 

o 1 

0 

0| 

0| 

0 j 

0 

0| 

01 

o 1 

0 

0| 

0| 

o 1 

0 

0| 

*1 

o 1 

2 

0| 

21 

o 1 

0 

0| 

01 

o 1 

0 

0| 

0| 

o 1 

0 

01 
0| 
0| 
0| 
0| 
0| 
0| 
01 
0| 
0| 
0 I 
0 I 
0| 
0| 
0| 
0| 


0 

0 

0 

2 

1 

3 

3 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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task  name  | fatigue |aentjtrain | stress (phys^strain (preciseness |cog_attent | response^chaing | a t t en t i on_s pan | sleep | schedule | boredom 


drair_gen  | 

0| 

M 

0| 

0| 

0| 

p rog r a n_eg»t  | 

°l 

0| 

0| 

0\ 

0| 

weather  | 

0| 

0| 

0| 

0| 

0| 

console^ops  | 

1 1 

3 t 

2| 

0| 

<M 

eir^traf f ic  | 

3| 

3 1 

31 

°l 

3| 

pilot_training  | 

0| 

11 

M 

2| 

equip_diag  | 

0| 

0| 

0| 

21 

M 

language  train  | 

0| 

0| 

0| 

°l 

0| 

f or*_till_out  j 

0| 

0| 

0| 

0| 

sw  design  | 

°l 

0| 

0| 

0| 

<U 

cargo  loading  | 

0| 

0| 

«l 

0| 

0| 

leadership  | 

0| 

0| 

0| 

0| 

0| 

surgery  1 

3| 

3 1 

3 1 

3| 

4 1 

aedical^diag  j 

0| 

3 1 

3 1 

0| 

*1 

accounting  j 

0| 

°l 

«l 

0| 

*1 

protocol  des  | 

0| 

0| 

«l 

0| 

0| 

€ I 


0 1 

0 I 

0 1 

0| 

0 

0 1 

0| 

0 1 

0 | 

0 

0 

0| 

° i 

0 1 

0 

4 1 

<>l 

4 1 

0 1 

1 1 

3 

4 1 

2| 

4 1 

0 j 

41 

0 

2 1 

3 1 

2 j 

0 1 

<>l 

2 

0 1 

0| 

0 1 

0 1 

°l 

1 

0 1 

0| 

0 1 

0 1 

°l 

0 

o 1 

0| 

o I 

0 j 

0| 

0 

0 1 

0| 

0 1 

o t 

0| 

0 

0 1 

0| 

o i 

0 j 

<>l 

0 

0 1 

0| 

0 I 

0 1 

0| 

2 

4 1 

0| 

4 j 

0 1 

*1 

0 

o 1 

0| 

0 1 

0 j 

0| 

° i 

o 1 

0| 

0 j 

0 1 

0| 

0 

o 1 

0| 

0 1 

0 j 

0| 

0 

< 


AI VAX $ DUA1 : ( AI . KAT AX ] P E RSON AL_PH Y S 2 .LIS; 2 


task_name  J gene  ra lheal th | age | gender {height {weight 


drair  gen  | 

0| 

0 1 

0 1 

o 1 

0 

P r og r a ■_ngn t | 

0| 

0 1 

0 1 

0 1 

0 

weather  j 

0| 

0 1 

o 1 

0 1 

0 

console  ops  j 

31 

1 1 

0 j 

0 1 

0 

air  traffic  j 

3| 

1 1 

0 1 

o I 

0 

pilot  training  J 

3 1 

1 1 

0 1 

0 1 

1 

equip  diag  j 

0| 

o t 

0 j 

0 1 

0 

language  train  | 

0| 

o 1 

o 1 

0 1 

0 

fora  fill  out  | 

0| 

0 1 

0 1 

0 1 

0 

sw  design  | 

0! 

0 1 

0 1 

o 1 

0 

cargo  loading  | 

0| 

0 1 

0 j 

0 j 

0 

1 eade  cship  | 

01 

o 1 

0 I 

0 1 

0 

surgery  | 

3| 

1 1 

0 1 

0 1 

0 

aedical  diag  j 

0 1 

0 1 

0 I 

0 

accounting  | 

<>l 

0 j 

o 1 

o 1 

0 

protocol  des  j 

0| 

0 1 

0 1 

0 1 

0 
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task  name  |edu_| t rain_expert | in tro_ac curacy | prob_of_success | articulation  | kr  hp 


drair  gen  | 

3 I 

M 

0 1 

0| 

1 | 

1 

program  ngnt  | 

2 j 

H 

0 1 

0| 

3 | 

1 

waathar  | 

2 1 

*\ 

0 I 

3| 

i j 

1 

console  ops  | 

3 | 

*\ 

0 1 

H 

1 1 

2 

air  traffic  j 

0 1 

4I 

0 1 

4I 

3 1 

2 

pilot  training  { 

o 1 

4 1 

0 1 

u 

1 | 

2 

equip  diag  | 

0 j 

H 

0 j 

0| 

0 1 

1 

language_t rain  | 

4 1 

4 1 

1 1 

11 

4 1 

1 

f or*_f ill_out  j 

3 | 

M 

o 1 

0| 

2 1 

1 

sw  design  j 

3 j 

0| 

Q j 

0| 

2 j 

1 

ca  r go_loadi|ng  | 

1 | 

3| 

0 j 

<H 

1 0 | 

1 

leadership  | 

1 1 

2| 

3 1 

3 t 

4 1 

1 

surgery  j 

4 1 

4I 

0 1 

3 | 

1 ! 

2 

■edicaldiag  | 

4 1 

4I 

0 1 

3 | 

4 1 

1 

accounting  j 

4 1 

M 

0 1 

°l 

1 1 

1 

protocol  des  | 

4 1 

“I 

0 1 

<>| 

2 | 

1 
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AI V AX $ DU Al : [AI  . KATAX ] EQUI P JTOOLS 2 . LIS ; 


task  name 


| computer  | vehicles  | weaponry  s terns  | instruients  |notation  | tes  t equip 


drair^gen 
pr og  ra*_mg»t 
weat he r 
console^ops 
airtraffic 
pilottraining 
•quip_di ag 
languaget ra in 
£or«J  i ll_out 
s w design 
cargo^loading 
leadership 
surgery 
■edicaldi ag 
accounting 
pr otocol_des 


3 | 

0| 

0| 

01 

0| 

1| 

0| 

0| 

0| 

0| 

4 | 

0| 

0| 

2| 

4 1 

4 | 

0| 

0| 

0 1 

4 1 

4 | 

0| 

0| 

4 1 

2 1 

0 1 

4 | 

01 

4 1 

2 I 

0 1 

0| 

01 

*1 

3 1 

0 I 

0| 

01 

0| 

2 1 

2 | 

0| 

0| 

01 

M 

4 | 

0| 

0| 

01 

3 1 

0| 

0 1 

01 

0| 

2 1 

o 1 

0| 

01 

0| 

31 

01 

0| 

01 

4 1 

21 

01 

0| 

0| 

3 1 

3 1 

3 | 

0| 

0| 

o| 

4 1 

0| 

0| 

0| 

0| 

21 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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APPENDIX  I 


INITIAL  DATA  ANALYSES  FROM  FIRST  INTERVIEW 


10-JUN-1991  17:33 


AIVAXSDUAl : [ AI . KATAX 1 PV_EQUAL_COUNT . ; 1 


task  namel 

| task  name2 

| equal 

_count | similar 

count 

pilottraining 

| protocol  des 

1 

25 1 

55 

pilot_training 

jdrair  gen 

1 

28  | 

51 

pilot  training 

| form  fill  out 

1 

28 1 

54 

pilot  training 

| accounting 

1 

29 1 

53 

pilottraining 

(weather 

1 

32 1 

54 

pilot_training 

jsw  design 

1 

33 1 

54 

console  ops 

( form  fill  out 

| 

34 1 

67 

leadership 

j surgery 

34 1 

62 

equip_diag 

| pilot  training 

35 1 

56 

pilottraining 

| cargo  loading 

1 

35 1 

56 

drair_gen 

j surgery 

1 

36 1 

54 

surgery 

j accounting 

1 

36 1 

59 

console  ops 

j leadership 

1 

36 1 

69 

pilot  training 

| leadership 

1 

37 1 

61 

equip  diag 

| console  ops 

I 

37 1 

68 

console  ops 

jdrair  gin 

37  | 

69 

console  ops 

jsw  design 

1 

37 1 

61 

console  ops 

| language  train 

1 

37 1 

66 

pilot  training 

| program  mgmt 

1 

38 1 

57 

equip_diag 

j surgery 

1 

39 1 

63 

air_traf fic 

j form_fill  out 

1 

39 1 

58 

task  namel 

| task  name2 

| equal 

count [similar 

count 

weather 

| surgery 

1 

39 1 

64 

form  fill  out 

j surgery 

1 

39 1 

60 

console  ops 

j programjngmt 

1 

41 1 

66 

console  ops 

j cargo_loading 

1 

41 1 

74 

pilot_training 

j language  train 

1 

41 1 

68 

pilottraining 

j medical  iiag 

1 

42  | 

72 

air_traffic 

j drair_gen 

1 

43  | 

66 

pilottraining 

j surgery 

1 

44  j 

81 

equipdiag 

| medical  diag 

I 

44 1 

74 

air  traffic 

j leadership 

1 

44  [ 

67 

console  ops 

(accounting 

1 

44 1 

73 

surgery- 

j protocol  des 

1 

44 1 

61 

pilot_training 

j air_traflic 

1 

45  | 

82 

pilot  training 

(console  ops 

45 1 

78 

console  ops 

j protocol  des 

45 1 

69 

sw_design 

j surgery 

1 

45 1 

61 

cargo_loading 

j medical  diag 

1 

45 1 

73 

cargo_loading 

j surgery 

1 

45 1 

67 

medical_diag 

j accounting 

1 

46 1 

71 

language_ train 

j surgery 

1 

46 1 

67 

console  ops 

j surgery 

1 

46 1 

78 

task  namel 

| task  name2 

| equal 

count | similar 

count 

console  ops 

| medical  diag 

1 

46 1 

78 

form_fill_out 

j medical  diag 

1 

47 1 

77 

programjngmt 

j surgery 

1 

47 1 

67 

console  ops 

j weather 

47 1 

72 

equip  diag 

(air  traffic 

1 

47 1 

66 

air_traf f ic 

j medical  jiiag 

1 

47  | 

67 

air_traf fic 

j accounting 

I 

48 1 

64 
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AI VAXSDUAl : [AI. KATAX  j PU_EQUAL_COUNT . ; 1 


equip  diag 

| language  train  | 

48 1 

73 

air  traffic 

| cargo  loading  j 

48 1 

77 

air  traffic 

j protocol  des  j 

49 1 

67 

leadership 

j medical  diag  j 

50 1 

86 

equip  diag 

j protocol  des  | 

50 1 

72 

air  traffic 

j sw  design  j 

50 1 

65 

dr air  gen 

j medical  diag  j 

50 1 

81 

cargo  loading 

(leadership  j 

50  ( 

82 

air  traffic 

j language  train  j 

51 1 

68 

air  traffic 

[weather  | 

52 1 

70 

equip  diag 

[leadership  j 

52 1 

81 

weather 

[medical  diag  j 

52 1 

81 

equip_diag 

jdrair  gen  j 

54  ( 

88 

weather 

[ language_train  j 

54| 

77 

task,  namel 

| task_name2  | equal_count | similar 

count 

equip_diag 

|sw  design  | 

55 1 

77 

weather 

[leadership  | 

55 1 

77 

air  traffic 

| console  ops  j 

56 1 

77 

program  mgmt 

| medical  diag  j 

56 1 

81 

cargo  loading 

j language  train  j 

56( 

73 

medical  diag 

[protocol  des  I 

57 1 

88 

drair_gen 

j cargo  loading  j 

57 

79 

cargo_loading 

j protocol  des  j 

57 1 

80 

surgery 

[medical  diag  j 

57  | 

87 

air  traffic 

(program  mgmt  j 

57 1 

72 

sw  design 

(medical  diag  j 

57 1 

84 

language_  train 

[accounting  j 

58 1 

69 

equip  diag 

j cargo  loading  j 

58 1 

82 

equip  diag 

j form  Till  out  j 

59 1 

84 

air  traffic 

j surgery  ~ j 

59 1 

83 

cargo_loading 

jsw  design  j 

59 1 

78 

equip_diag 

j weather  j 

60 1 

79 

equip_diag 

j program  mgmt  j 

60 1 

85 

weather 

j cargo  loading  j 

60 1 

81 

language_train 

[medical  diag  j 

60 1 

83 

cargo  loading 

[ form_f ill  out  j 

61 1 

83 

task  namel 

| task_name2  | equal  count | similar 

count 

weather 

| program  mgmt  | 

61 1 

84 

equip_diag 

[accounting  j 

61 1 

88 

language_train 

[protocol  des  j 

62 1 

85 

program  mgmt 

j cargo  loading  j 

63 1 

92 

form  fill  out 

j language  train  j 

63 1 

82 

leadership 

j protocol  des  j 

64 1 

94 

leadership 

[accounting  j 

64 

88 

accounting 

j protocol  des  j 

65 1 

88 

drairgen 

[language  train  j 

65 1 

82 

weather 

[accounting  j 

66 1 

86 

weather 

jform  fill  out  | 

66 1 

85 

cargo  loading 

[accounting  j 

66 1 

89 

form  lill  out 

[leadership  j 

67 1 

98 

program  mgmt 

jdrair  gen  j 

67( 

91 

drair_gen 

[leadership  j 

67 1 

89 

weather 

j protocol  des  j 

67 1 

83 

weather 

[ sw_design  j 

68 1 

93 
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_AIVAXSDUA1 : [ AI 

sw_design 
form_f ill_out 
svdesign 
sw_design 

task  namel 


drair_gen 

prograra_mgmt 

language_train 

programjngmt 

programjngmt 

drair_gen 

drair_gen 

program_mgmt 

weather 

sv_design 

drair_gen 

sw_deslgn 

program  mgmt 

£orm_fiTl_out 

program  mgmt 


KATAX } PW_EQUAL_COUNT . ; 1 

[ leadership 
| protocol_des 
j language_train 
I accounting 

| task  name2 


| protocol_des 
| language_train 
| leadership 
j protocol_des 
| sw_design 
| accounting 
j sw_design 
j form_f ill_out 
jdrair  gen 
| form_Iill_out 
j form_fill  out 
j protocoljles 
j accounting 
j accounting 
j leadership 


1 68 1 

90 

1 68| 

94 

1 69 1 

87 

1 69 1 

89 

| equal_count | similar 

count 

1 69! 

93 

1 70 1 

87 

1 70 1 

90 

1 71 1 

92 

1 72| 

93 

1 72| 

101 

| 72 1 

95 

I 73| 

95 

1 73| 

94 

! 73| 

94 

1 74 1 

90 

1 76 1 

96 

1 76 1 

93 

! 76 1 

95 

1 78| 

99 
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AIVAXSDUAl : ( AI .KATAX] PW_SIMILAR_COUNT. ; 1 


task  namel 

task_name2  (equal  count | similar 

count 

pilottraining 

drair  gen  | 

28  | 

51 

pilottraining 

accounting  j 

29 1 

53 

pilot  training 

form  fill  out  | 

28  | 

54 

pilot_t raining 

veather  j 

32 1 

54 

pilot_training 

sw  design  j 

33 1 

54 

drair  gen 

surgery  j 

36  j 

54 

pilot_training 

protocol  des  j 

25 1 

55 

pilot_training 

cargo  loading  j 

35  | 

56 

equip_diag 

pilot  training  i 

35 1 

56 

pilot_training 

program  mgmt  j 

38 1 

57 

air_traf f ic 

form  fill  out  | 

39 1 

58 

surgery 

accounting  j 

36 1 

59 

form_fill_out 

surgery  j 

39 1 

60 

pilot_training 

leadership  j 

37 1 

61 

console  ops 

sw  design  1 

37 1 

61 

sw  design 

surgery  j 

45 1 

61 

surgery 

protocol  des  j 

441 

61 

leadership 

surgery  j 

34  ( 

62 

equipdiag 

surgery  | 

39 1 

63 

weather 

surgery  j 

39 1 

64 

airtraffic 

accounting  j 

48 1 

64 

task  namel 

taskjname2  | equal  count  1 similar 

_count 

air_traffic 

sw  design  | 

50 1 

65 

console  ops 

language  train  j 

37 1 

66 

console  ops 

program  mgmt 

41 1 

66 

equip_dlag 

air  traffic  1 

47  | 

66 

air_traf fic 

drair  gen  j 

43 1 

66 

air_traf fic 

medical  diag  j 

47 1 

67 

air_traf fic 

protocol  des  j 

49  ( 

67 

cargo_loading 

surgery  j 

45 1 

67 

console  ops 

form  fill  out  j 

34 1 

67 

languagetrain 

surgery  j 

46 1 

67 

airtraf f ic 

leadership  j 

44 1 

67 

program_mgmt 

surgery  | 

47  | 

67 

pilot_training 

language  train  i 

41 1 

68 

air_traf f ic 

language  train  | 

51 1 

68 

equip_diag 

console  ops  j 

37  | 

68 

console  ops 

protocol  des  [ 

45 1 

69 

console  ops 

leadership  j 

36 1 

69 

language_ train 

accounting  | 

58 1 

69 

console  ops 

drair  gen  j 

37 1 

69 

airtraff ic 

weather  j 

52  | 

70 

medical  diag 

accounting  | 

461 

71 

task_namel 

task_name2  | equal_count (similar 

count 

equipdiag 

protocol  des  | 

50 1 

72 

pilot_training 

medical  Jiag  j 

42 1 

72 

air_traffic 

program  mgmt  j 

57| 

72 

console  ops 

weather  j 

47 1 

72 

equip  diag 

language  train  j 

48 1 

73 

console  ops 

accounting  j 

44 

73 

cargoloading 

language_ train  j 

56! 

73 
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cargo  loading 

consoIe_ops 

equip_diag 

air_traf f ic 

veather 

weather 

equip_diag 

form_f ill_out 

air_traff ic 

cargoloading 

console_ops 

pilot  training 

consoIe_ops 

equip_diag 

tasknamel 

drair_gen 

cargoloading 

pilot_training 

weather 

weather 

drair_gen 

program  mgmt 

equipdlag 

cargo_loading 

drair_gen 

equip_diag 

pilot  training 

form_?ill  out 

cargo_loa3ing 

weather 

air_traffic 

language_train 

weather 

sv_design 

equip_diag 

languagetrain 

task_namel 

equipdiag 

weather 

weather 

leadership 

program_mgmt 

surgery 

sw  design 

medical  diag 

accounting 

equip_diag 

leadership 

equip_diag 

drair_gen 

sw_design 

cargo_loading 

sw_design 

drair_gen 


| cargo_loading 
I protocol_des 
I surgery 
j cargo_loading 
| medical_diag 
| medical_diag 
(medical  diag 
| leadership 
| leadership 
I language_train 
|cargo_loading 
jair_traffic 
(language  train 
|form_filI  out 
| protocol_<Ies 
I surgery 
j medical_diag 
| program_mgmt 
| medical  diag 
jform_fiIl  out 
| protocol_<Ies 

( task  name2 


57 

79 

57 

80 

44 

81 

60 

81 

52 

81 

50 

81 

56 

81 

52 

81 

50 

82 

65 

82 

58 1 

82 

45| 

82 

63 1 

82 

61 1 

83 

67 1 

83 

59 1 

83 

60 1 

83 

61 1 

84 

57  | 

84 

59 1 

84 

62 1 

85 

int  | 

similar  count 

60 1 

85 

66 1 

85 

66 1 

86 

50 1 

86 

70 1 

87 

57 1 

87 

69 1 

87 

57 1 

88 

65 1 

88 

54 1 

88 

64 1 

88 

61 1 

88 

67 1 

89 

69 1 

89 

66 1 

89 

68 1 

90 

74 1 

90 

| program  mgmt 
( form_fiIl_out 
| accounting 
j medical_diag 
I language  train 
| medical_cTiag 
( language_train 
i protocol_des 
| protocol_des 
j drair_gen 
(accounting 
j accounting 
| leadership 
(accounting 
(accounting 
j leadership 
jform  fill  out 


equal 
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language  train 
program  mgmt 
program  mgmt 
program  mgmt 

task  namel 

(leadership  | 70 ( 90 
jdrair_gen  j 67 | 91 
j protocol_des  j 71 j 92 
| cargo_loading  j 63  j 92 

| task._name2  | equal  count  | similar  count 

weather 

|sv  design  | 

68 1 

93 

drair  gen 

(protocol  des  j 

69 1 

93 

program_mgmt 

jsv_design  j 

72 1 

93 

program  mgmt 

(accounting  j 

76 1 

93 

veather 

j drair  gen  j 

7 3 ( 

94 

leadership 

j protocol  des  j 

64 1 

94 

sv  design 

j form  fill  out  j 

73 1 

94 

form  fill  out 

j protocol  <Ies  j 

68 1 

94 

drair  gen 

| sv  design  j 

72 

95 

program  mgmt 

j form  fill  out  j 

7 3 1 

95 

form  fill  out 

(accounting  j 

7 6 1 

95 

sv  design 

j protocol  des  j 

76 1 

96 

form  fill  out 

{leadership  j 

67 1 

98 

program  mgmt 

{leadership  j 

78  ( 

99 

drair_gen 

{accounting  j 

72  ( 

101 
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attrib_narae  | zero_cnt | one_cnt | tvo_cnt | three_cnt | four  cnt 


casesexamples 

sub  goals 

higK_low_tech 

procedures 

quant  qual 

facts 

compare 

categorize 

acq_and_pres 

goal_vs_data 

written 

edu_train_expert 

temporal 

principles 

case_based 

kr_hp 

decompose 

communicates 

form_proc_alg 

verbal  ~ 

learn 


0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
1 
1 
1 
1 
1 
2 
2 
2 j 
21 


i| 
5| 
0| 
II 
2| 
II 
11 
0[ 
6| 
2| 
3 1 
1| 
3 1 
2 I 
HI 
1| 
0| 
II 
1| 
2 1 


7| 
3 1 
9| 
3 1 
9| 
31 
2 I 
6| 
3 1 
5| 
1| 
5| 
3 1 
2 I 
4| 
2 I 
4I 
0! 
2 1 
0| 
5| 


4I 
3| 
2 1 
1| 
6| 
3 1 
HI 
5| 

4I 
5| 
5| 
8| 
II 
5| 
8| 
2 1 
5| 
5| 
4I 
3 1 
3 1 


1 

9 

0 

12 

0 

8 

2 

4 
9 
0 
8 
0 

10 

5 
1 
0 
5 
9 
7 

10 

4 


attrib  name 


zero  cnt 


one_cnt | tvo_cnt | three_cnt | four_cnt 


choose 

interpret 

uncertainty 

notation 

requests 

analyze 

propadeutics 

constraints 

transmits 

informs 

estimate 

answers 

itemize 

branching 

historical 

articulation 

computer 

inductive 

plan 

directs 

supervise 


2 

2 

3 

3 

3 

3 

3 

3 

4 
4 
4 
4 

4 

5 
5 
5 
5 
5 

5 

6 
6 


2 I 
2 1 
5| 
1| 
2 I 
11 
5| 
5| 
0| 
0| 
1| 
0| 
2 I 
4I 
0| 
2 I 
H 

2 I 
11 
II 

3 I 


4I 
2 1 
2 1 
*1 
5| 
3 I 
6| 
3 1 
5| 
4! 
4I 
3 1 
3 1 
1| 
3 1 
11 
4! 
6| 
0| 
2 I 
0| 


3 1 
5 1 
4I 
2 1 
4I 
2 1 
1| 
2 I 
5| 
11 
4I 
5| 
3 1 
2 1 
2 1 
4I 
11 
2 1 
3 1 
2 1 
2 1 


5 

5 
2 

6 
2 
7 
1 
3 
2 
7 

3 

4 
4 
4 
6 

4 

5 
1 
7 
5 
5 


attrib  name 


zero  cnt 


one_cnt | two_cnt | three_cnt | four  cnt 


induce  | 6 
model_based  j 6 
generalize  | 6 
prob_of_success  j 6 
dynamism  j 7 
ret  to_def  | 7 
monitor  I 7 


2 1 6|  0|  2 

3 1 3 1 3|  1 

2 1 3 1 5|  0 

2 1 1|  4 1 3 
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schedule:  2 

boredom:  1 

general_health : 2 

age : 0 

gender : 0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  1 

kr_hp:  1 

computer : 3 

vehicles:  2 

weapon_sy stems : 0 

instruments:  0 

notation:  2 

test_equip:  0 

advises:  4 

answers : 2 

communicates:  3 

directs:  3 

indicates:  0 

informs : 0 

instructs:  0 

requests:  2 

transmits:  0 

supervises:  4 

negotiates:  0 

express_movement : 2 

interp_movement : 0 

namel:  equip_diag 

name2 : sw_design 

eq_count:  55 

sim  count:  77 


statistical:  0 

spatial:  1 

temporal:  3 

analogical : 0 

case_based:  1 

model_based:  2 

^ mathematical:  1 

deductive:  2 

inductive : 2 

means_ends : 4 

sub_goals : 1 

gen__and_test : 3 

goal_vs_data : 2 

cover_and_dif f : 0 

prop_and_ref : 4 

acq_and_pres : 2 

f orm_proc__alg : 2 

decompose:  1 

recombine:  3 

generalize:  2 

specialize : 0 

ret_to_def:  0 

visual:  1 

verbal:  4 

auditory:  1 

kinesthetic:  0 

written:  1 

instrumentation : 4 

mm_interf ace : 1 

databases : 1 

historical : 3 

propadeutics : 1 

branching : 2 

dynamism:  0 

constraints:  2 

uncertainty:  0 

high_low_tech : 1 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  2 

gross:  2 

repetitive:  4 

categorize : 1 

calculate : 0 

code : 2 

computerize:  2 

interpolate:  0 

itemize:  2 

learn : 1 

tabulate:  0 

translate : 4 

analyze:  2 

deduce : 2 

induce:  2 

choose:  2 

compare : 1 

compute : 0 

estimate:  3 

integrate : 4 

plan:  3 

supervise:  1 

monitor:  0 

interpret:  0 


C-3 


facts:  1 

principles:  2 

procedures:  2 

analogs:  2 

cases_examples : 1 

percep_speed : 0 

search_rec_inf o : 0 

id_ob j_act_events : 0 

scan_display : 0 

acceleration:  0 

confinement:  0 

isolation:  0 

contaminants:  2 

electricity:  3 

lighting:  0 

magnetism:  0 

noise:  4 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  3 

preciseness:  0 

cog_attent:  0 

response_chaining : 0 

attention_span : 1 

sleep:  0 

schedule:  2 

boredom:  3 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight:  0 

edu_train_expert : 2 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  3 

kr_hp : 1 

computer : 1 

vehicles:  0 

weapon_systems : 0 

instruments:  0 

notation:  0 

test_equip:  0 

advises:  4 

answers : 0 

communicates:  3 

directs:  0 

indicates:  0 

informs : 0 

instructs:  0 

requests:  0 

transmits:  2 

supervises:  0 

negotiates:  4 

express_movement : 0 

interp_movement : 0 

namel : equip_diag 

name2 : form_f ill_out 

eq_count : 59 

sim  count:  84 


statistical:  0 


spatial:  1 

temporal:  1 

analogical:  2 

case_based:  1 

model_based : 1 

mathematical:  2 

deductive:  2 

inductive : 2 

means_ends:  2 

sub__goals : 1 

gen_and_test : 2 

goal_vs_data : 0 

cover_and_dif f : 
prop__and_ref : 2 

acq__and_jpres : 2 

f orm_j?roc_alg : 
decompose : 0 

recombine:  2 

generalize:  3 

specialize:  0 

ret_to_def:  1 

visual:  1 

verbal:  3 

auditory:  1 

kinesthetic:  0 

written:  1 

instrumentation : 
mm_interf ace : 1 

databases:  0 

historical:  1 

propadeutics : 0 

branching : 1 

dynamism:  0 

constraints : 1 

uncertainty:  1 

high_low_tech : 
quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  2 

gross:  2 

repetitive : 4 

categorize:  0 

calculate:  2 

code : 0 

computerize : 0 

interpolate:  2 

itemize:  2 

learn : 0 

tabulate:  1 

translate : 0 

analyze:  0 

deduce:  2 

induce:  2 

choose:  4 

compare : 1 

compute : 2 

estimate:  2 

integrate : 1 

plan:  0 

supervise : 0 

monitor : 0 

interpret : 2 

facts:  0 


0 


principles:  1 

procedures:  0 

analogs : 0 

cases_examples : 1 

percep_speed:  0 

search_rec_inf o : 0 

id_ob j_act_events : 
scan_display : 0 

acceleration:  0 

confinement:  0 

isolation:  0 

contaminants:  2 

electricity:  3 

lighting:  0 

magnetism:  0 

noise:  4 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  3 

preciseness:  0 

cog_attent:  0 

response_chaining : 0 

attention_span : 1 

sleep:  0 

schedule:  2 

boredom:  3 

general_health : 0 

age : 0 

gender:  0 

height:  0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  0 

kr_hp : 1 

computer : 1 

vehicles:  0 

weapon_systems : 0 

instruments : 0 

notation:  1 

test_equip:  0 

advises:  4 

answers:  3 

communicates:  3 

directs:  0 

indicates:  3 

informs : 2 

instructs:  0 

requests:  2 

transmits:  4 

supervises:  0 

negotiates:  4 

express_movement : 0 

interp_movement : 0 

namel : equip_diag 

name2 : language_train 

eq_count:  48 

sim  count:  73 


statistical:  0 

spatial:  1 


temporal:  1 

analogical:  3 

case_based:  1 

model_based:  1 

mathematical:  0 

deductive : 0 

inductive : 2 

means_ends : 1 

sub_goals : 1 

gen__and_test : 0 

goal_vs_data : 2 

cover_and_dif f : 
Pr°P_and__ref : 2 

acq_ and_pres : 2 

f orm_jproc_alg : 
decompose : 1 

recombine:  3 

generalize:  3 

specialize:  1 

ret_to_def:  1 

visual:  1 

verbal:  4 

auditory:  1 

kinesthetic:  0 

written;  1 
instrumentation : 
mm_interf ace : 1 

databases:  1 

historical:  3 

propadeutics : 0 

branching:  2 

dynamism:  4 

constraints:  0 

uncertainty : 2 

high_low_tech : 
quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  2 

gross : 2 

repetitive : 4 

categorize:  1 

calculate:  0 

code:  0 

computerize:  0 

interpolate:  0 

itemize:  3 

learn:  1 

tabulate:  0 

translate : 4 

analyze:  1 

deduce : 0 

induce : 2 

choose:  1 

compare:  1 

compute:  0 

estimate : 0 

integrate : 0 

plan:  4 

supervise:  4 

monitor:  4 

interpret:  1 

facts:  2 

principles:  3 
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procedures:  2 

analogs : 3 

cases_examples : 2 

percep_speed:  0 

search_rec_inf o : 0 

id_ob j_act_events : 0 

scan_display : 0 

acceleration:  0 

confinement:  0 

isolation:  0 

contaminants:  2 

electricity : 3 

lighting:  0 

magnetism:  0 

noise:  4 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  3 

preciseness : 0 

cog_attent:  0 

response_chaining : 0 

attention_span : 1 

sleep:  0 

schedule:  2 

boredom:  3 

general_health : 0 

age : 0 

gender:  0 

height:  0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 4 

prob_of_success : 2 

articulation : 4 

kr_hp:  1 

computer:  1 

vehicles:  0 

weapon_systems : 0 

instruments:  0 

notation:  4 

test_equip:  0 

advises:  2 

answers : 4 

communicates:  4 

directs:  4 

indicates : 4 

informs : 4 

instructs : 4 

requests:  4 

transmits : 4 

supervises:  4 

negotiates : 0 

express_movement : 3 

interp_movement : 3 

namel:  equip_diag 

name2 : leadership 

eq_count:  52 

sim  count:  81 


statistical:  0 

spatial:  1 

temporal:  1 


analogical:  1 

case_based:  1 

model_based:  1 

mathematical:  0 

deductive:  2 

inductive : 2 

means_ends : 0 

sub_goals : 1 

ge  n_a  nd_t  e s t : 2 

goal_vs_data : 2 

cover_and_dif f : 0 

prop_and_ref : 3 

acq_and_pres : 1 

form_proc_alg : 1 

decompose:  0 

recombine : 1 

generalize:  1 

specialize:  0 

ret_to_def:  1 

visual:  1 

verbal:  4 

auditory:  1 

kinesthetic:  0 

written:  0 

instrumentation:  4 

mm_interface : 1 

databases : 1 

historical:  0 

propadeutics : 0 

branching:  0 

dynamism:  0 

constraints:  3 

uncertainty:  0 

high_low_tech : 0 

quant_qual : 1 

complex_^cont : 0 

compound:  0 

reflex:  0 

s impl e_di sc:  0 

fine:  2 

gross:  2 

repetitive:  4 

categorize:  1 

calculate:  2 

code:  0 

computerize:  1 

interpolate:  0 

itemize:  0 

learn:  1 

tabulate:  0 

translate:  0 

analyze:  2 

deduce : 2 

induce:  1 

choose:  3 

compare : 1 

compute:  1 

estimate:  1 

integrate:  2 

plan;  3 
supervise:  4 

monitor:  0 

interpret : 3 

facts:  1 

principles:  2 

procedures:  0 


analogs  5 1 

cases_examples : 1 

percep_speed:  0 

search_rec_inf o : 0 

id_ob  j_act_events  : 0 

scan_display : 0 

acceleration:  0 

confinement:  0 

isolation : 0 

contaminants:  2 

electricity : 3 

lighting:  0 

magnetism:  0 

noise:  4 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain : 3 

preciseness:  0 

cog_attent : 0 

response_chaining : 0 

attention_span : 1 

sleep:  0 

schedule:  2 

boredom:  3 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 4 

prob_of_success : 3 

articulation:  4 

kr_hp : 1 

computer : 1 

vehicles:  0 

weapon_systems : 0 

instruments:  0 

notation:  2 

test_equip:  0 

advises:  0 

answers : 2 

communicates:  4 

directs:  2 

indicates : 2 

informs:  4 

instructs : 4 

requests:  3 

transmits : 3 

supervises:  3 

negotiates:  3 

express_movement : 3 

interp_movement : 3 

namel:  equip_diag 

name2 : surgery 

eq_count : 39 

sim  count:  63 


statistical:  0 

spatial:  3 

temporal : 3 

analogical : 1 


case_based:  0 

modeljbased:  2 

mathematical:  0 

deductive:  1 

inductive:  2 

means_ends : 4 

sub_goals : 1 

gen_and_test : 1 

goal_vs_data : 2 

cover_and_dif f : 
prop_and_ref : 0 

acq_and__pres : 1 

form_proc_alg : 0 

decompose : 1 

recombine:  1 

generalize:  1 

specialize:  3 

ret_to_def:  0 

visual:  3 

verbal:  4 

auditory:  1 

kinesthetic:  4 

written:  1 

instrumentation : 
mm_interface : 0 

databases:  1 

historical:  1 

propadeutics : 1 

branching:  3 

dynamism:  2 

constraints:  1 

uncertainty:  1 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 4 

compound:  2 

reflex:  0 

simple_disc:  4 

fine:  2 

gross:  2 

repetitive:  0 

categorize:  1 

calculate:  0 

code:  0 

computerize : 0 

interpolate:  0 

itemize:  3 

learn:  3 

tabulate:  0 

translate:  0 

analyze:  2 

deduce : 2 

induce:  2 

choose:  4 

compare : 1 

compute : 0 

estimate:  3 

integrate:  0 

plan:  4 

supervise:  4 

monitor:  4 

interpret : 1 

facts:  2 

principles:  3 

procedures : 0 

analogs:  0 


cases_examples : 1 

percep_speed:  2 

search_rec_info : 0 

id__ob  j_act_events  : 4 

scan_display : 0 

acceleration : 0 

confinement : 2 

isolation : 0 

contaminants : 1 

electricity : 3 

lighting:  2 

magnetism:  0 

noise:  4 

fatigue:  3 

mental_strain : 3 

stress:  4 

phys_strain : 0 

preciseness:  4 

cog__attent : 4 

response_chaining : 2 

attention_span : 3 

sleep:  2 

schedule:  2 

boredom:  2 

general_health : 0 

age : 0 

gender : 0 

height : 0 

weight : 0 

edu__train_expert : 2 _ 

intro_accuracy : 0 

prob_of_success : 4 

articulation:  1 

kr_hp : 3 

computer:  3 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  0 

test_equip:  3 

advises : 1 

answers:  3 

communicates:  3 

directs:  4 

indicates:  3 

informs : 2 

instructs : 0 

requests:  2 

transmits : 0 

supervises:  4 

negotiates:  0 

express_movement : 2 

interp_movement : 0 

namel:  equip_diag 

name2 : medical_diag 

eq^count : 44 

sim  count:  74 


statistical:  0 

spatial:  2 

temporal:  3 

analogical : 0 

case  based:  1 


model_based:  1 

mathematical:  0 

deductive:  2 

inductive:  2 

means_ends : 1 

sub_goals : 1 

gen_and_test : 1 

goal_vs_data : 1 

cover_and_dif f : 
prop_and_ref : 4 

acq_and_pres : 2 

f orm_proc_alg : 2 

decompose:  0 

recombine:  1 

generalize:  2 

specialize:  2 

ret_to_def:  1 

visual:  3 

verbal:  4 

auditory:  1 

kinesthetic:  1 

written:  1 

instrumentation : 
mm_interf ace : 1 

databases:  1 

historical:  1 

propadeutics : 1 

branching:  3 

dynamism:  0 

constraints:  2 

uncertainty:  2 

high_low_tech : 1 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  1 

gross:  1 

repetitive:  4 

categorize:  1 

calculate:  0 

code:  0 

computerize:  0 

interpolate:  0 

itemize:  1 

learn:  3 

tabulate:  0 

translate:  3 

analyze:  2 

deduce:  2 

induce:  2 

choose:  4 

compare : 2 

compute : 0 

estimate:  2 

integrate:  0 

plan:  1 

supervise:  0 

monitor:  2 

interpret : 1 

facts:  2 

principles:  3 

procedures:  0 

analogs : 0 

cases_examples : 2 


percep_speed:  0 

search_rec_inf o : 0 

id_ob  j_act_events  : 0 

scan_display : 0 

acceleration:  0 

confinement : 0 

isolation : 0 

contaminants:  2 

electricity : 3 

lighting:  0 

magnetism:  0 

noise:  4 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  3 

preciseness:  0 

cog_ attent : 0 

response_chaining : 0 

attention_span : 1 

sleep:  1 

schedule:  0 

boredom:  2 

general_health : 0 

age : 0 

gender:  0 

height:  0 

weight : 0 

edu_train_expert : 2 

intro_accuracy : 3 

prob_of_success : 4 

articulation : 3 

kr_hp:  1 

computer:  3 

vehicles:  0 

weapon__systems : 0 

instruments : 2 

notation:  0 

test_equip:  2 

advises:  0 

answers:  4 

communicates:  4 

directs : 4 

indicates : 2 

informs:  4 

instructs : 1 

requests:  1 

transmits:  2 

supervises:  1 

negotiates : 0 

express_movement : 2 

interp_movement : 3 

namel:  equip_diag 

name2 : accounting 

eq_count : 61 

sim  count:  88 


statistical : 0 

spatial : 1 

temporal:  2 

analogical : 0 

case_based:  0 

model  based:  1 


mathematical:  4 

deductive:  2 

inductive:  0 

means_ends : 0 

sub_goals : 0 

gen_and_test : 1 

goal_vs_data : 1 

cover_and_dif f : 
prop_and_ref : 0 

acq_and_pres : 2 

form_proc_alg:  0 

decompose:  1 

recombine:  1 

generalize:  0 

specialize:  3 

ret_to_def : 1 

visual:  1 

verbal:  3 

auditory:  1 

kinesthetic:  0 

written:  1 

instrumentation : 
mm_interf ace : 3 

databases:  3 

historical:  1 

propadeutics : 0 

branching:  1 

dynamism:  1 

constraints:  2 

uncertainty:  0 

high_low__tech : 0 

quant_qual : 2 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  2 

gross:  2 

repetitive:  4 

categorize:  1 

calculate:  4 

code : 0 

computerize:  4 

interpolate:  0 

itemize:  1 

learn:  2 

tabulate:  4 

translate:  1 

analyze:  2 

deduce:  2 

induce:  0 

choose:  1 

compare:  1 

compute : 4 

estimate:  0 

integrate:  3 

plan:  0 

supervise:  0 

monitor : 0 

interpret : 1 

facts:  1 

principles:  1 

procedures:  0 

analogs : 0 

cases_examples : 1 

percep_speed:  0 


search_rec_inf o : 0 

id__ob  j__act_events : 0 

scan_display : 0 

acceleration : 0 

confinement : 0 

isolation : 0 

contaminants : 2 

electricity:  3 

lighting:  0 

magnetism:  0 

noise:  4 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain : 3 

preciseness : . 4 
cog_attent:  0 

response_chaining : 0 

attention_span : 1 

sleep:  0 

schedule : 2 

boredom:  3 

general_health : 0 

age : 0 

gender:  0 

height:  0 

weight:  0 

edu_train_expert : 2 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  0 

kr_hp : 1 

computer:  1 

vehicles:  0 

weapon_systems : 0 

instruments : 0 

notation:  1 

test_equip:  0 

advises : 4 

answers:  2 

communicates:  4 

directs:  0 

indicates:  0 

informs : 2 

instructs : 0 

requests:  1 

transmits : 3 

supervises:  0 

negotiates:  .0 

express_movement : 0 

interp_movement : 0 

namel:  equip_diag 

name2 : protocol_des 

eq_count:  50 

sim  count:  72 


statistical : 2 

spatial:  0 

temporal : 3 

analogical : 0 

case_based:  2 

model_based:  1 

mathematical:  2 


deductive:  2 

inductive:  3 

means_ends:  0 

sub_goals : 1 

gen_and_test : 3 

goal_vs_data : 2 

cover_and_dif f : 0 

prop_and_ref : 4 

acq_and_j?res : 2 

form_proc_alg : 3 

decompose:  1 

recombine:  2 

generalize:  3 

specialize:  0 

ret_to_def:  0 

visual:  1 

verbal:  4 

auditory:  1 

kinesthetic:  0 

written:  0 

instrumentation:  2 

mm_interf ace : 1 

databases : 1 

historical:  1 

propadeutics : 2 

branching:  0 

dynamism:  0 

constraints:  3 

uncertainty : 1 

high_low_tech : 1 

quant_qual : 0 

complex_cont : 0 

compound;  0 
reflex:  0 

s impl  e_di s c : 0 

fine:  2 

gross:  2 

repetitive:  4 

categorize:  1 

calculate : 3 

code:  0 

computerize : 0 

interpolate:  1 

itemize:  1 

learn:  1 

tabulate:  0 

translate:  4 

analyze:  2 

deduce : 2 

induce:  2 

choose:  4 

compare : 1 

compute : 3 

estimate:  2 

integrate:  3 

plan:  4 

supervise:  1 

monitor:  0 

interpret : 1 

facts:  1 

principles:  2 

procedures : 0 

analogs : 0 

cases_examples : 3 

percep_speed:  0 

search  rec  info:  0 


id_ob  j__ac t_e vent  s : 0 

scan__display : 0 

acceleration:  0 

confinement : 0 

isolation : 0 

contaminants : 2 

electricity : 3 

lighting:  0 

magnetism:  0 

noise:  4 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  3 

preciseness : 0 

cog_attent : 0 

response_chaining : 0 

attention_span : 1 

sleep:  0 

schedule:  2 

boredom:  3 

general_health : 0 

age : 0 

gender:  0 

height:  0 

weight:  0 

edu_train_expert : 2 

intro_accuracy : 0 

prob_ of_success : 4 

articulation:  3 

kr_hp : 1 

computer:  1 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  4 

test_ equip:  1 

advises : 4 

answers : 3 

communicates:  4 

directs:  3 

indicates : 0 

informs:  3 

instructs : 0 

requests:  2 

transmits : 3 

supervises:  3 

negotiates:  0 

express_movement : 0 

interp_movement : 0 

namel : pilot_training 

name2 : air_traffic 

eq_count : 45 

sim  count:  82 


statistical : 0 

spatial:  0 

temporal:  0 

analogical : 3 

case_based:  2 

model_based:  2 

mathematical:  0 

deductive : 4 


inductive:  1 

means_ends : 1 

sub_goals : 0 

gen_and_test : 0 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : 0 

acq_and__pres : 2 

form_proc_alg:  4 

decompose:  1 

recombine:  0 

generalize:  2 

specialize:  3 

ret_to_def:  1 

visual:  0 

verbal:  1 

auditory:  2 

kinesthetic:  4 

written:  1 

instrumentation:  0 

mm_interf ace : 2 

databases:  0 

historical:  0 

propadeutics : 1 

branching:  0 

dynamism:  0 
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cases_examples : 1 

percep_speed:  4 

search_rec_inf o : 
id_obj_act_e  vents  : 
scan_display : 4 

acceleration:  0 

confinement:  1 

isolation : 0 

contaminants:  0 

electricity:  0 

lighting:  1 

magnetism:  0 

noise:  4 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_ strain:  1 

preciseness:  1 

cog_attent:  4 

response_chaining : 
attention_span : 3 

sleep:  0 

schedule:  2 

boredom:  2 

general__health : 4 

age : 1 

gender:  0 

height : 0 

weight:  2 

edu_train_expert : 
intro_accuracy : 0 

prob_of_success : 
articulation:  2 


0 


kr_hp : 1 

computer:  0 

vehicles:  0 

weapon_systems : 
instruments:  3 

notation:  1 

test_equip:  1 

advises : 2 

answers : 1 

communicates:  0 

directs:  1 

indicates:  1 

informs : 0 

instructs:  0 

requests:  1 

transmits:  1 

supervises:  0 

negotiates:  0 

express_movement : 3 

interp_movement : 0 

namel : console_ops 

name2 : cargo_loading 

eq_count:  41 

sim  count:  74 


statistical:  0 

spatial:  4 

temporal:  0 

analogical:  0 

case_based:  1 

model_based : 1 

mathematical : 1 

deductive:  3 

inductive : 1 

means_ends : 3 

sub_goals : 3 

gen_and_test : 0 

goal_vs_data : 1 

cover_and_dif f : 3 

prop_and_ref : 0 

acq_and_pres : 1 

f orm_proc_alg : 0 

decompose : 0 

recombine:  1 

generalize:  0 

specialize:  0 

ret_to_def:  2 

visual:  1 

verbal:  4 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  4 

mm_interf ace : 4 

databases : 0 

historical:  2 

propadeutics : 1 

branching:  1 

dynamism:  3 

constraints:  3 

uncertainty:  3 

high_low_tech : 1 

quant_qual : 0 


complex_cont : 0 

compound:  0 

reflex:  3 

simple_disc:  4 

fine:  4 

gross : 0 

repetitive:  4 

categorize:  1 

calculate : 1 

code : 0 

computerize:  0 

interpolate : 2 

itemize:  4 

learn : 2 

tabulate:  0 

translate : 2 

analyze:  3 

deduce:  3 

induce:  0 

choose:  0 

compare : 2 

compute : 1 

estimate:  1 

integrate:  3 

plan:  0 

supervise : 2 

monitor : 0 

interpret : 3 

facts:  0 

principles : 3 

procedures : 2 

analogs:  0 

cases_examples : 0 

percep_speed:  4 

, ^ search_rec_inf  o : 4 

id_obj_act_e  vents  : 4 

scan_display : 4 

acceleration:  1 

confinement : 1 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  1 

magnetism:  0 

noise:  1 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  0 

preciseness:  1 

cog_attent:  4 

response_chaining : 1 

attention_span : 3 

sleep:  0 

schedule:  2 

boredom:  0 

general_health : 2 

age : 1 

gender:  0 

height:  0 

weight:  2 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 3 

articulation:  1 

kr_hp:  1 


0 


computer : 4 

vehicles:  2 

weapon_systems : 
instruments : 3 

notation:  0 

test_equip:  1 

advises : 2 

answers:  1 

communicates:  1 

directs : 2 

indicates : 1 

informs : 4 

instructs:  0 

requests:  1 

transmits:  3 

supervises:  4 

negotiates:  0 

express_movement : 1 

interp_movement : 0 

namel:  console_ops 

name2 : sw_design 

eq_count : 37 

sim  count:  61 


statistical:  0 

spatial:  0 

temporal:  0 

analogical:  0 

case_based:  0 

model_based:  1 

mathematical:  2 

deductive : 4 

inductive : 1 

means_ends : 4 

sub_goals : 3 

gen_and_test : 4 

goal_vs_data : 1 

cover_and_dif f : 3 

prop_and_ref : 4 

acq_and_pres : 2 

form_proc_alg:  2 

decompose:  2 

recombine : 4 

generalize:  2 

specialize:  2 

ret_to_def:  1 

visual:  4 

verbal : 0 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  4 

mm_interf ace : 4 

databases : 0 

historical:  2 

propadeutics : 2 

branching:  3 

dynamism:  4 

constraints:  3 

uncertainty:  2 

high_low_tech : 0 

quant_qual : 1 

complex_cont : 0 


compound:  0 

reflex:  3 

simple_disc:  4 

fine:  4 

gross:  0 

repetitive:  4 

categorize:  2 

calculate:  3 

code:  2 

computerize:  2 

interpolate:  4 

itemize:  1 

learn:  2 

tabulate:  0 

translate:  2 

analyze:  0 

deduce : 4 

induce:  2 

choose:  0 

compare : 0 

compute : 3 

estimate:  0 

integrate:  1 

plan:  1 

supervise:  0 

monitor:  4 

interpret : 0 

facts:  1 

principles:  2 

procedures:  2 

analogs : 2 

cases_examples : 0 

percep_speed:  4 

search_rec_inf o : 
id_ob j_act_events : 
scan_display : 4 

acceleration:  0 

confinement : 1 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  1 

magnetism:  0 

noise:  4 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  1 

preciseness:  1 

cog_attent:  4 

response_chaining : 
attention_span : 3 

sleep:  0 

schedule:  2 

boredom:  2 

general_health : 4 

age : 1 

gender:  0 

height : 0 

weight : 2 

edu_train_e Xpert : 
intro_accuracy : 0 

prob_of_success : 
articulation:  1 

kr_hp : 1 

computer : 0 


0 


vehicles:  0 

weapon_systems : 
instruments:  3 

notation:  2 

test_equip:  1 

advises:  2 

answers : 3 

communicates:  1 

directs:  1 

indicates : 1 

informs : 4 

instructs:  0 

requests:  3 

transmits:  1 

supervises:  0 

negotiates:  4 

express_movement : 3 

interp_movement : 0 

namel : console_ops 

name2 : form_f ill_out 

eq_count:  34 

sim  count:  67 


statistical:  0 

spatial:  0 

temporal:  2 

analogical:  2 

case_based:  0 

model_based : 2 

mathematical : 1 

deductive : 4 

inductive : 1 

means_ends : 2 

sub_goals : 3 

gen_and__test : 3 

goal_vs_data:  1 

cover_and_dif f : 0 

prop_and_ref : 2 

acq_and_pres : 2 

form_proc_alg:  0 

decompose:  1 

recombine : 3 

generalize:  3 

specialize:  2 

ret_to_def : 2 

visual:  4 

verbal:  1 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  4 

mm_interface:  4 

databases : 1 

historical:  0 

propadeutics : 1 

branching:  0 

dynamism:  4 

constraints:  2 

uncertainty:  3 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 0 

compound : 0 


reflex:  3 

simple_disc : 4 

fine:  4 

gross:  0 

repetitive : 4 

categorize : 1 

calculate : 1 

code : 0 

computerize:  0 

interpolate:  2 

itemize:  1 

learn : 3 

tabulate:  1 

translate : 2 

analyze:  2 

deduce:  4 

induce:  2 

choose:  2 

compare : 0 

compute:  1 

estimate:  1 

integrate:  2 

plan:  4 

supervise : 1 

monitor:  4 

interpret : 2 

facts:  2 

principles : 1 

procedures:  0 

analogs : 0 

cases_examples : 0 

percep_speed : 4 

search_rec_info : 4 

id_obj_act_e vents : 4 

^ scan_display : 4 

acceleration:  0 

confinement : 1 

isolation : 0 

contaminants:  0 

electricity:  0 

lighting:  1 

magnetism:  0 

noise:  4 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  1 

preciseness : 1 

cog_attent:  4 

response_chaining : 1 

attention__span : 3 

sleep:  0 

schedule:  2 

boredom:  2 

general_health:  4 

age : 1 

gender:  0 

height : 0 

weight:  2 

edu_train_expert : 1 

intro_accuracy : 0 

^prob_of_success : 3 

'"articulation:  2 

kr_hp : 1 

computer:  0 

vehicles:  0 


0 


weapon_systems : 
instruments:  3 

notation:  1 

test_equip:  1 

advises:  2 

answers:  0 

communicates:  1 

directs:  1 

indicates:  2 

informs : 2 

instructs:  0 

requests:  1 

transmits:  1 

supervises:  0 

negotiates:  4 

express_movement : 3 

interp_movement : 0 

namel : console_ops 

name2 : language_train 

eq_count : 37 

sim  count:  66 


statistical:  0 

spatial:  0 

temporal:  4 

analogical:  3 

case_based:  0 

model_based:  2 

mathematical:  3 

deductive : 2 

inductive:  1 

means_ends : 1 

sub_goals : 3 

gen_and_test : 1 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : 2 

acq_and_pres : 2 

form_proc_alg : 4 

decompose:  2 

recombine:  4 

generalize:  3 

specialize:  1 

ret_to_def:  0 

visual:  4 

verbal : 0 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  0 

mm_interface:  4 

databases:  0 

historical:  2 

propadeutics : 1 

branching:  3 

dynamism:  0 

constraints:  1 

uncertainty:  0 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  3 


simple_disc:  4 

fine:  4 

gross:  0 

repetitive:  4 

categorize : 2 

calculate : 3 

code:  0 

computerize : 0 

interpolate:  4 

itemize:  0 

learn:  2 

tabulate:  0 

translate : 2 

analyze:  1 

deduce:  2 

induce:  2 

choose:  1 

compare:  0 

compute:  3 

estimate:  3 

integrate : 3 

plan:  0 

supervise : 3 

monitor:  0 

interpret : 1 

facts:  0 

principles:  3 

procedures : 2 

analogs : 3 

cases_examples : 1 

percep_speed : 4 

search_rec_inf o : 
id_obj__act_e  vents  : 
scan_display : 4 

-acceleration:  0 

confinement : 1 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  1 

magnetism:  0 

noise:  4 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  1 

preciseness:  1 

cog_attent:  4 

response_chaining : 
attention_span : 3 

sleep:  0 

schedule:  2 

boredom:  2 

general_health : 4 

age : 1 

gender:  0 

height : 0 

weight : 2 

edu_train_expert : 
intro_accuracy : 4 

orob_of_success : 
Articulation:  2 

kr__hp:  1 

computer:  2 

vehicles:  0 

weapon_systems  0 


instruments:  3 

notation:  2 

test_equip:  1 

advises : 0 

answers : 1 

communicates:  0 

directs:  3 

indicates:  3 

informs:  0 

instructs:  4 

requests:  1 

transmits:  1 

supervises:  4 

negotiates:  0 

express_movement : 0 

interp_movement : 3 

namel:  console_ops 

name2 : leadership 

eq_count:  36 

sim  count:  69 


statistical:  0 

spatial:  0 

temporal:  2. 

analogical:  1 

case_based:  0 

model_based:  2 

mat  hemat i ca 1 : 3 

deductive:  4 

inductive : 1 

means_ends : 0 

sub_goals : 1 

gen_and_test : 3 

goal_vs_data : 1 

cover_and_dif f : 3 

prop_and_ref : 3 

acq_and_pres : 1 

form_proc_alg:  1 

decompose:  1 

recombine:  0 

generalize:  1 

specialize:  2 

ret_to_def:  0 

visual:  4 

verbal:  0 

auditory:  0. 

kinesthetic:  0 

written:  0 

instrumentation:  4 

mm_interface:  4 

databases:  2 

historical:  1 

propadeutics : 1 

branching:  1 

dynamism:  4 

constraints:  4 

uncertainty:  2 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  3 

s imple_di sc:  4 


fine:  4 

gross:  0 

repetitive : 4 

categorize : 2 

calculate : 1 

code : 0 

computerize:  1 

interpolate:  4 

itemize:  3 

learn:  4 

tabulate:  0 

translate : 2 

analyze:  0 

deduce:  4 

induce:  1 

choose:  1 

compare : 0 

compute : 2 

estimate:  2 

integrate : 1 

plan:  1 

supervise : 3 

monitor : 4 

interpret:  3 

facts:  1 

principles:  2 

procedures:  0 

analogs : 1 

cases_examples : 0 

percep_speed:  4 

search_rec_inf o : 
id_ob j_act_events : 
scan_display : 4 

acceleration:  0 

confinement:  1 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  1 

magnetism:  0 

noise:  4 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  1 

preciseness:  1 

cog__attent : 4 

response_chaining : 
attention_span : 3 

sleep:  0 

schedule:  2 

boredom:  2 

general_health : 4 

age : 1 

gender : 0 

height : 0 

weight:  2 

edu_train_e xpert : 
intro_accuracy : 4 

prob__of_success  : 
articulation:  2 

kr_hp:  1 

computer:  2 

vehicles:  0 

weapon__systems : 0 

instruments:  3 


notation:  0 

test_equip:  1 

advises:  2 

answers : 1 

communicates:  0 

directs:  1 

indicates:  1 

informs:  0 

instructs:  4 

requests:  0 

transmits:  0 

supervises:  3 

negotiates:  3 

express_movement : 0 

interp_movement : 3 

namel:  console_ops 

name2 : surgery 

eq_count : 46 

sim  count:  78 


statistical:  0 

spatial:  4 

temporal:  0 

analogical:  1 

case_based:  1 

model_based:  1 

mathematical:  3 

deductive : 1 

inductive:  1 

means_ends : 4 

sub_goals : 3 

gen_and_test : 0 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : 0 

acq_and__pres : 1 

f orm_proc_alg : 0 

decompose:  2 

recombine:  0 

generalize:  1 

specialize:  1 

ret_to_def:  1 

visual:  0 

verbal:  0 

auditory:  2 

kinesthetic:  4 

written:  1 

instrumentation : 0 

mm_interf ace : 3 

databases : 0 

historical:  2 

propadeutics : 2 

branching:  4 

dynamism:  2 

constraints:  2 

uncertainty:  1 

high_low_tech : 0 

quant_qual : 1 

complex_cont : 4 

compound:  2 

reflex:  3 

simple_disc : 0 

fine:  0 


gross:  4 

repetitive : 0 

categorize : 0 

calculate:  3 

code:  0 

computerize:  0 

interpolate:  4 

itemize:  0 

learn:  0 

tabulate:  0 

translate : 2 

analyze:  0 

deduce:  0 

induce:  2 

choose:  2 

compare : 0 

compute:  3 

estimate:  0 

integrate:  3 

plan:  0 

supervise:  3 

monitor:  0 

interpret : 1 

facts:  0 

principles:  3 

procedures:  0 

analogs : 0 

cases_examples : 0 

percep_speed:  2 

search_rec_inf o : 4 

id_ob j_act_ events : 0 

scan_display : 4 

acceleration:  0 

confinement:  1 

isolation : 0 

contaminants:  1 

electricity : 0 

lighting:  1 

magnetism:  0 

noise:  4 

fatigue:  2 

mental_strain : 1 

stress:  1 

phys_strain : 2 

preciseness:  3 

cog_attent:  0 

response_chaining : 1 

attention_span : 1 

sleep:  2 

schedule:  2 

boredom:  1 

general_health : 4 

age : 1 

gender:  0 

height:  0 

weight:  2 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 1 

articulation:  1 

kr_hp : 1 

. computer : 4 

vehicles:  0 

weapon_systems : 0 

instruments:  1 

notation:  2 


test_equip:  2 

advises:  1 

answers : 0 

communicates:  1 

directs:  3 

indicates:  2 

informs : 2 

instructs : 0 

requests:  1 

transmits:  3 

supervises:  4 

negotiates:  0 

express_movement : 1 

interp_movement : 0 

namel : console_ops 

name2 : medical_diag 

eq_count : 46 

sim  count:  78 


statistical:  0 

spatial:  3 

temporal:  0 

analogical:  0 

case_based:  0 

model_based:  0 

mathematical:  3 

deductive : 0 

inductive:  1 

means_ends : 1 

sub_goals : 1 

gen_and_test : 0 

goal_vs_data : 0 

cover_and_diff : 2 

prop_and_ref : 4 

acq_and__pres : 2 

form__proc_alg:  2 

decompose:  1 

recombine : 0 

generalize:  2 

specialize:  0 

ret_to_def:  2 

visual:  0 

verbal:  0 

auditory:  2 

kinesthetic:  1 

written:  1 

instrumentation:  0 

mm_interf ace : 4 

databases:  0 

historical:  2 

propadeutics : 2 

branching:  4 

dynamism:  4 

constraints:  3 

uncertainty:  0 

high_low_tech : 0 

quant_qual : 0 

complex_cont : 0 

compound : 0 

reflex:  3 

simple_disc:  4 

fine:  3 

gross:  1 


repetitive : 4 

categorize:  0 

calculate : 3 

code:  0 

computerize:  0 

interpolate:  4 

itemize:  2 

learn:  0 

tabulate:  0 

translate : 1 

analyze:  0 

deduce : 0 

induce:  2 

choose:  2 

compare : 1 

compute : 3 

estimate:  1 

integrate : 3 

plan:  3 

supervise : 1 

monitor:  2 

interpret:  1 

facts:  0 

principles:  3 

procedures:  0 

analogs : 0 

cases_examples : 1 

per cep__ speed:  4 

search_rec_info : 
id_ob j_act_event s : 
scan_display : 4 

acceleration:  0 

confinement:  1 

isolation:  0 

contaminant  s : 0 

electricity : 0 

lighting:  1 

magnetism:  0 

noise:  4 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  1 

preciseness:  1 

cog_attent : 4 

response_chaining : 
attention_span : 3 

sleep:  1 

schedule:  0 

boredom:  1 

general_health : 4 

age : 1 

gender:  0 

height:  0 

weight:  2 

edu__train_e xpert : 
intro_accuracy : 3 

prob_of_success : 
articulation:  1 

kr_hp:  1 

computer:  4 

vehicles:  0 

weapon_systems : 
instruments:  1 

notation:  2 

test_equip:  1 


0 


advises : 2 

answers : 1 

communicates : 0 

directs:  3 

indicates : 1 

informs : 0 

instructs:  1 

requests:  2 

transmits:  1 

supervises:  1 

negotiates:  0 

express_movement : 1 

interp_movement : 3 

namel:  console_ops 

name2 : accounting 

eq_count:  44 

sim  count:  73 


statistical:  0 

spatial:  0 

temporal:  1 

analogical:  0 

case_based:  1 

model_based:  2 

mathematical:  1 

deductive : 4 

inductive:  1 

means_ends : 0 

sub_goals : 2 

gen_and_test : 0 

goal_vs_data:  0 

cover_and_dif f : 3 

prop_and_ref : 0 

acq_and_pres : 2 

form_proc_alg:  0 

decompose:  0 

recombine:  2 

generalize:  0 

specialize:  1 

ret_to_def ; 0 

visual:  4 

verbal:  1 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  4 

mm_interf ace : 0 

databases:  4 

historical:  2 

propadeutics : 1 

branching:  0 

dynamism:  3 

constraints:  3 

uncertainty:  2 

high_low_tech : 1 

quant_qual : 2 

complex_cont : 0 

compound:  0 

reflex:  3 

simple_disc:  4 

fine:  4 

gross:  0 

repetitive : 4 


acq_and_jpres : 0 

form_proc_alg : 0 

decompose:  0 

recombine:  3 

generalize:  1 

specialize : 3 

ret_to_def : 1 

visual:  4 

verbal:  0 

auditory:  2 

kinesthetic:  4 

written:  2 

instrumentation : 4 

mm__interf  ace : 4 

databases:  1 

historical : 2 

propadeutics : 2 

branching : 4 

dynamism:  4 

constraints:  2 

uncertainty : 4 

high_low_tech : 0 

quant_qual:  0 

complex_cont : 2 

compound:  0 

reflex:  0 

simple_disc:  4 

fine:  3 

gross:  3 

repetitive:  1 

categorize:  0 

calculate : 2 

code:  0 

computerize:  0 

interpolate:  2 

itemize:  1 

learn : 1 

tabulate:  1 

translate : 0 

analyze:  1 

deduce : 3 

induce:  1 

choose:  1 

compare:  1 

compute:  2 

estimate:  2 

integrate:  1 

plan:  3 

supervise:  3 

monitor : 4 

interpret : 2 

facts:  1 

principles:  1 

procedures:  0 

analogs:  3 

cases_examples : 1 

percep_speed:  4 

search_rec_info : 3 

id_obj_act_e vents : 3 

scan_display : 4 

acceleration:  3 

confinement : 4 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 


magnetism:  0 

noise:  4 

fatigue:  3 

mental_strain : 3 

stress:  3 

phys_strain:  1 

preciseness:  3 

cog_attent:  4 

response_chaining : 1 

attention_span : 3 

sleep:  0 

schedule:  3 

boredom:  0 

general_health : 4 

age : 3 

gender : 2 

height:  1 

weight : 3 

edu_train_expert : 1 

intro_accuracy : 3 

prob_of_success : 3 

articulation:  4 

k r_hp : 2 

computer:  4 

vehicles:  4 

weapon_systems : 0 

instruments:  4 

notation:  1 

test_equip:  0 

advises:  4 

answers : 1 

communicates:  0 

directs : 4 

indicates:  1 

informs:  2 

instructs:  4 

requests:  1 

transmits : 2 

supervises:  3 

negotiates:  4 

express_movement : 0 

interp_movement : 2 

namel:  pilot_training 

name2 : language_train 

eq_count : 41 

sim  count:  68 


statistical:  0 

spatial:  4 

temporal:  4 

analogical:  0 

case_based:  0 

model_based:  3 

mathematical:  0 

deductive:  2 

inductive:  1 

means_ends : 2 

sub_goals : 0 

gen_and_test : 1 

goal_vs_data:  2 

cover_and_dif f : 2 

prop_and_ref : 2 

acq_and_pres : 0 


f orm_j?roc_alg : 4 

decompose:  1 

recombine : 4 

generalize:  1 

specialize:  2 

ret_to_def:  1 

visual:  4 

verbal:  1 

auditory:  2 

kinesthetic : 4 

written:  2 

instrumentation : 
mm_interf ace : 4 

databases:  0 

historical:  0 

propadeutics : 2 

branching:  i 

dynamism:  0 

constraints:  1 

uncertainty : 1 

high_low_tech ; 0 

quant_qual:  0 

complex__cont : 2 

compound:  0 

reflex:  0 

simple_disc:  4 

fine:  3 

gross:  3 

repetitive:  1 

categorize : 1 

calculate : 0 

code : 0 

computerize:  0 

interpolate:  0 

itemize:  2 

learn:  2 

tabulate:  0 

translate : 4 

analyze:  0 

deduce : 1 

induce:  1 

choose:  2 

compare : 1 

compute:  0 

estimate:  0 

integrate : 2 

plan:  1 

supervise:  1 

monitor:  0 ■ 

interpret:  1 

facts:  3 

principles:  3 

procedures:  2 

analogs : 0 

cases__examples : 
percep_speed:  4 

search_rec_inf o : 
id_ob  j_act_e  vent  s 
scan_display : 4 

acceleration:  3 

confinement:  4 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 


noise:  4 

fatigue:  3 

mental_strain : 3 

stress:  3 

phys_strain:  1 

preciseness : 3 

cog_attent : 4 

response_cha!ning : 1 

attention_span : 3 

sleep:  0 

schedule:  3 

boredom:  0 

general_health : 4 

age : 3 

gender : 2 

height : 1 

weight:  3 

edu_train_expert : 1 

intro_accuracy : 1 

prob_of_success : 1 

articulation:  0 

kr_hp : 2 

computer:  2 

vehicles:  4 

weapon_systems : 0 

instruments : 4 

notation : 2 

test_equip:  0 

advises : 2 

answers : 0 

communicates:  1 

directs:  0 

indicates : 0 

informs : 0 

instructs:  0 

requests:  1 

transmits:  2 

supervises:  1 

negotiates:  0 

express_movement : 3 

interp_movement : 1 

namel : pilot_training 

name2 : leadership 

eq_count:  37 

sim  count:  61 


statistical:  0 

spatial : 4 

temporal:  2 

analogical:  2 

case_based:  0 

model_based:  3 

mathematical:  0 

deductive : 4 

inductive:  1 

means_ends : 3 

sub_goals : 2 

gen_a  nd_t  e s t : 3 
goal_vs_data : 2 

cover_and_dif f : 0 

prop_and_ref : 3 

acq_and_j>res : 1 

form_proc_alg : 1 


decompose : 0 

recombine:  0 

generalize:  1 

specialize : 3 

ret_to_def : 1 

visual:  4 

verbal : 1 

auditory:  2 

kinesthetic : 4 

written:  1 

instrumentation : 
mm_interf ace : 4 

databases : 2 

historical:  3 

propadeutics : 2 

branching:  3 

dynamism:  4 

constraints : 4 

uncertainty:  3 

high_low_tech : 0 

quant_qual : 0 

complex_cont : 2 

compound:  0 

reflex:  0 

simple_disc:  4 

fine:  3 

gross:  3 

repetitive:  1 

categorize : 1 

calculate:  2 

code:  0 

computerize:  1 

interpolate:  0 

itemize:  1 

learn : 0 

tabulate:  0 

translate:  0 

analyze:  1 

deduce : 3 

induce:  0 

choose:  0 

compare:  1 

compute:  1 

estimate:  1 

integrate : 0 

plan:  0 

supervise:  1 

monitor:  4 

interpret : 3 

facts:  2 

principles:  2 

procedures:  0 

analogs : 2 

cases_examples : 
percep_speed:  4 

search_rec_inf o : 
id_ob  j_ac  t_e  vent  s 
scan_display : 4 

acceleration:  3 

confinement:  4 

isolation : 0 

, contaminants : 0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  4 


fatigue:  3 

mental_strain : 3 

stress:  3 

phys_strain:  1 

preciseness:  3 

cog_attent : 4 

response_chaining : 1 

attention_span : 3 

sleep:  0 

schedule:  3 

boredom:  0 

general_health : 4 

age : 3 

gender : 2 

height:  1 

weight:  3 

edu_train_expert : 2 

intro_accuracy : 1 

prob_of_success : 0 

articulation:  0 

kr_hp:  2 

computer:  2 

vehicles:  4 

weapon_systems : 0 

instruments:  4 

notation:  0 

test_equip:  0 

advises:  0 

answers:  2 

communicates:  1 

directs:  2 

indicates : 2 

informs:  0 

instructs:  0 

requests:  0 

transmits:  1 

supervises:  0 

negotiates:  3 

express_movement : 3 

interp_movement : 1 

namel:  pilot_training 

name2 : surgery 

eq_count:  44 

sim  count:  81 


statistical:  0 

spatial:  0 

temporal:  0 

analogical:  2 

case_based:  1 

model_based : 0 

mathematical:  0 

deductive : 1 

inductive:  1 

means_ends : 1 

sub_goals : 0 

gen_and_test : 0 

goal_vs_data:  2 

cover_and_dif f : 2 

prop_and_ref : 0 

acq_and_pres : 1 

f orm_proc_alg : 0 

decompose : 1 


recombine:  0 

generalize:  1 

specialize:  0 

ret_to_def : 0 

visual:  0 

verbal:  1 

auditory:  0 

kinesthetic : 0 

written:  2 

instrumentation : 0 

mm_interf ace : 3 

databases:  0 

historical : 4 

propadeutics : 1 

branching:  0 

dynamism:  2 

constraints : 2 

uncertainty : 2 

high_low_tech : 1 

quant_qual : 0 

complex_cont : 2 

compound:  2 

reflex:  0 

simple_disc:  0 

fine:  1 

gross:  1 

repetitive:  3 

categorize:  1 

calculate:  0 

code:  0 

computerize:  0 

interpolate:  0 

itemize:  2 

learn:  4 

tabulate:  0 

translate : 0 

analyze:  1 

deduce : 1 

induce:  1 

choose:  1 

compare:  1 

compute:  0 

estimate:  3 

integrate : 2 

plan:  1 

supervise : 1 

monitor:  0 

interpret : 1 

facts:  3 

principles : 3 

procedures:  0 

analogs:  3 

cases_examples : 1 

percep_speed:  2 

search_rec_inf o : 3 

id_ob  j__act_events : 
scan__display : 4 

acceleration:  3 

confinement:  2 

isolation : 0 

contaminants:  1 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  4 

fatigue:  0 


mental_strain : 0 

stress:  1 

phys_strain:  2 

preciseness:  1 

cog_attent:  0 

response_chaining:  1 

attention_span : 1 

sleep:  2 

schedule:  1 

boredom:  1 

general_health : 4 

age : 3 

gender : 2 

height : 1 

weight : 3 

edu_train_expert : 0 

intro_accuracy : 3 

prob_of_success : 1 

articulation:  3 

kr_hp : 0 

computer : 0 

vehicles:  4 

weapon_systems : 0 

instruments:  0 

notation:  2 

test_equip:  3 

advises:  1 

answers:  1 

communicates:  0 

directs:  0 

indicates : 1 

informs : 2 

instructs:  4 

requests:  1 

transmits : 2 

supervises:  1 

negotiates:  0 

express_movement : 2 

interp_movement : 2 

namel:  pilot_training 

name2 : medical_diag 

eq_count:  42 

sim  count:  72 


statistical:  0 

spatial:  1 

temporal : 0 

analogical:  3 

case_based:  0 

modeljbased:  1 

mathematical:  0 

deductive:  0 

inductive:  1 

means_ends:  2 

sub_goals:  2 

gen_and_test : 0 

goal_vs_data : 1 

cover_and_dif f : 1 

prop_and_ref : 4 

acq_and_pres : 0 

form_proc_alg:  2 

decompose : 0 

recombine:  0 


generalize:  0 

specialize:  1 

ret_to_def:  1 

visual:  0 

verbal:  1 

auditory:  0 

kinesthetic : 3 

written:  2 

instrumentation : 0 

mm__interf  ace : 4 

databases:  0 

historical : 4 

propadeutics : 1 

branching : 0 

dynamism:  4 

constraints:  3 

uncertainty:  1 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 2 

compound:  0 

reflex:  0 

simple_disc : 4 

fine:  2 

gross:  2 

repetitive:  1 

categorize:  1 

calculate:  0 

code:  0 

computerize:  0 

interpolate:  0 

itemize:  0 

learn:  4 

tabulate:  0 

translate : 3 

analyze:  1 

deduce:  1 

induce:  1 

choose:  1 

compare : 2 

compute : 0 

estimate:  2 

integrate : 2 

plan:  2 

supervise:  3 

monitor:  2 

interpret : 1 

facts:  3 

principles:  3 

procedures:  0 

analogs : 3 

cases_examples : 0 

percep_speed:  4 

search_rec_info : 3 

id_obj_act_e vents : 
scan__display : 4 

acceleration : 3 

confinement : 4 

isolation : 0 

contaminant  s : 0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  4 

fatigue:  2 

mental  strain: 


1 


1 


stress:  0 

phys_strain:  1 

preciseness:  3 

cog_attent:  4 

response_chaining : 
attention_span : 3 

sleep:  1 

schedule:  1 

boredom:  1 

general_health : 4 

age : 3 

gender : 2 

height : 1 

weight : 3 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 1 

articulation:  1 

kr_hp:  2 

computer:  0 

vehicles:  4 

weapon_systems : 0 

instruments:  2 

notation:  2 

test_equip:  2 

advises:  0 

answers : 0 

communicates:  1 

directs:  0 

indicates : 2 

informs:  0 

instructs : 3 

requests:  2 

transmits:  0 

supervises:  2 

negotiates:  0 

express_movement : 2 

interp_movement : 1 

namel : pilot_training 

name2 : accounting 

eq_count : 29 

sim  count:  53 


statistical:  0 

spatial:  4 

temporal:  1 

analogical:  3 

case_based:  1 

model_based:  3 

mathematical:  4 

deductive : 4 

inductive : 1 

means_ends : 3 

sub_goals : 1 

gen_and_test : 0 

goal_vs_data : 1 

cover_and_dif f : 0 
prop_and_ref : 0 

acq_and_pres : 0 

f orm_proc_alg : 0 

decompose:  1 

recombine : 2 

generalize:  2 


specialize:  0 

ret_to_def : 1 

visual:  4 

verbal:  0 

auditory:  2 

kinesthetic : 4 

written:  2 

instrumentation : 
mm_interf ace : 0 

databases : 4 

historical : 4 

propadeutics : 2 

branching:  4 

dynamism:  3 

constraints:  3 

uncertainty:  3 

high_low_tech : 0 

quant_qual : 1 

complex_cont : 2 

compound:  0 

reflex:  0 

simple_disc:  4 

fine:  3 

gross:  3 

repetitive:  1 

categorize:  1 

calculate : 4 

code : 0 

computerize:  4 

interpolate : 0 

itemize:  2 

learn : 3 

tabulate:  4 

translate : 1 

analyze:  3 

deduce : 3 

induce:  1 

choose:  2 

compare:  1 

compute:  4 

estimate:  0 

integrate:  1 

plan:  3 

supervise : 3 

monitor:  4 

interpret:  1 

facts:  0 

principles : 1 

procedures : 0 

analogs : 3 

cases__examples : 
percep_speed:  4 

search_rec_inf o : 
id_ob  j_ac t_e vent  s 
scan_display : 4 

acceleration:  3 

confinement:  4 

isolation : 0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  4 

fatigue:  3 

mental_strain : 
stress:  3 


3 


1 


phys_strain:  1 

preciseness:  1 

cog  attent:  4 

response_chaining:  1 

attention_span : 
sleep:  0 

schedule:  3 

3 

boredom:  0 

general_health : 

4 

age : 3 

gender:  2 

height : 1 

weight : 3 

edu_train_e xpert 

: 0 

intro_accuracy : 

3 

prob_of_success : 
articulation:  4 

3 

kr_hp:  2 

computer : 4 

vehicles:  4 

weapon  systems: 
instruments:  4 

notation:  1 

0 

test_equip:  0 

advises:  4 

answers : 2 

communicates:  1 

directs:  4 

indicates:  4 

informs:  2 

instructs : 4 

requests:  2 

transmits:  1 

supervises:  3 

negotiates:  0 

express_movement : 

: 0 

interp_movement : 

2 

namel : pilot  training 

name2 : protocol 

eq_count:  25 

sim  count:  55 

_des 

statistical:  2 

spatial:  3 

temporal:  0 

analogical:  3 

case_based:  1 

model_based:  1 

mathematical:  2 

deductive : 4 

inductive:  2 

means_ends : 3 

sub_goals : 0 

gen_and_test : 4 

goa  l_vs_dat  a : 2 

cover_and_dif f : 0 

prop_and_ref : 4 

acq_and_pres : 0 

form_proc_alg : 3 

decompose : 1 

recombine : 3 

generalize:  1 

specialize:  3 


ret_to_def:  0 

visual:  4 

verbal:  1 

auditory:  2 

kinesthetic:  4 

written:  1 

w instrumentation:  2 

mm_interf ace : 4 

databases : 0 

historical : 4 

propadeutics : 4 

branching : 3 

dynamism:  4 

constraints:  4 

uncertainty:  4 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 2 

compound:  0 

reflex:  0 

simple_disc:  4 

fine:  3 

gross:  3 

repetitive:  1 

categorize : 1 

calculate:  3 

code:  0 

computerize:  0 

interpolate : 1 

itemize:  2 

learn:  2 

tabulate:  0 

translate:  4 

analyze:  3 

deduce:  3 

induce:  1 

choose:  1 

compare : 1 

compute:  3 

estimate:  2 

integrate:  1 

plan:  1 

supervise:  2 

monitor:  4 

interpret:  1 

facts:  2 

principles : 2 

procedures : 0 

analogs : 3 

cases_examples : 1 

percep_speed:  4 

search_rec_inf o : 3 

id_ob  j_act_events  : 3 

scan_display : 4 

acceleration:  3 

confinement : 4 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise : 4 

fatigue:  3 

mental_strain : 3 

stress:  3 

phys_strain:  1 


1 


preciseness : 3 

cog_attent:  4 

response_chaining : 
attention_span : 3 

sleep:  0 

schedule:  3 

boredom:  0 

general_health:  4 

age : 3 

gender:  2 

height:  1 

weight : 3 

edu_train_expert : 0 

intro_accuracy : 3 

prob_of_success : 1 

articulation:  1 

kr_hp : 2 

computer:  2 

vehicles:  4 

weapon_systems : 0 

instruments:  0 

notation:  2 

test_equip:  1 

advises:  4 

answers:  1 

communicates:  1 

directs:  1 

indicates:  4 

informs : 1 

instructs:  4 

requests:  1 

transmits:  1 

supervises:  0 

negotiates:  0 

express_movement : 0 

interp_movement : 2 

namel : air_traffic 

name2 : console_ops 

eq_count:  56 

sim  count : 77 


statistical:  0 

spatial:  4 

temporal : 0 

analogical:  0 

case_based:  2 

modeljbased:  1 

mathematical:  3 

deductive:  4 

inductive:  1 

means_ends : 4 

sub_goals : 3 

gen_and_test : 0 

goal_vs_data:  0 

cover_and_diff : 2 

prop_and_ref : 0 

acq_and_pres : 0 

formjproc_alg:  4 

decompose:  2 

recombine:  0 

generalize:  0 

specialize:  2 

ret  to  def:  0 


visual:  0 

verbal:  0 

auditory:  0 

kinesthetic : 0 

written:  2 

instrumentation : 
mm_interf  ace : 2 

databases : 0 

historical:  2 

propadeutics : 2 

branching : 4 

dynamism:  0 

constraints:  0 

uncertainty : 2 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  2 

s imple_disc:  2 

fine:  2 

gross:  0 

repetitive:  2 

categorize:  2 

calculate : 3 

code : 0 

computerize:  0 

interpolate:  1 

itemize:  3 

learn : 1 

tabulate:  0 

translate:  2 

analyze:  4 

deduce : 4 

induce : 0 

choose:  2 

compare : 0 

compute : 3 

estimate:  1 

integrate:  3 

plan:  0 

supervise : 3 

monitor:  0 

interpret : 2 

facts:  0 

principles:  2 

procedures : 0 

analogs : 0 

cases_examples : 
percep_speed:  0 

search_rec_inf o : 
id_ob  j_ac  t_e  vent  s 
scan_display : 0 

acceleration:  0 

confinement:  2 

isolation : 0 

contaminants : 0 

electricity : 0 

lighting:  1 

magnetism:  0 

noise:  1 

fatigue:  3 

mental_strain : 2 

stress:  1 

phys_strain : 1 

preciseness:  1 


cog_attent : 0 

response_chaining:  2 

attention_span : 1 

sleep:  4 

schedule:  2 

boredom:  2 

general_health : 0 

age : 1 

gender:  0 

height : 0 

weight:  2 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  1 

kr_hp : 0 

computer:  4 

vehicles:  0 

weapon_systems : 0 

instruments:  1 

notation:  2 

test_equip:  1 

advises:  1 

answers : 3 

communicates:  0 

directs:  3 

indicates:  1 

informs : 0 

instructs:  0 

requests:  1 

transmits:  0 

supervises:  4 

negotiates:  0 

express_movement : 3 

interp_movement : 0 

namel : air_traffic 

name2 : weather 

eq_count : 52 

sim  count:  70 


statistical:  1 

spatial:  0 

temporal:  0 

analogical:  0 

case_based:  1 

model_based:  3 

mathematical:  2 

deductive : 0 

inductive:  3 

means_ends : 4 

sub_goals : 1 

gen_and_test : 0 

goal_vs_data : 0 

cover_and_diff : 1 

prop_and_ref : 0 

acq_and_pres : 2 

form_proc_alg : 3 

decompose : 4 

recombine:  3 

generalize:  3 

specialize:  0 

ret_to_def:  0 

visual:  0 


verbal : 3 

auditory:  0 

kinesthetic:  0 

written:  2 

instrumentation : 
mm_interf ace : 2 

databases : 0 

historical:  2 

propadeutics : 0 

branching : 0 

dynamism:  0 

constraints:  3 

uncertainty:  3 

high_low_tech : 1 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  1 

simple_disc:  2 

fine:  2 

gross:  0 

repetitive : 2 

categorize : 1 

calculate : 2 

code : 0 

computerize:  0 

interpolate : 1 

itemize:  3 

learn:  1 

tabulate:  0 

translate : 2 

analyze:  4 

deduce : 0 

induce:  4 

choose:  2 

compare : 0 

compute:  1 

estimate:  0 

integrate:  4 

plan : 4 

supervise:  4 

monitor:  1 

interpret:  2 

facts:  0 

principles : 1 

procedures:  0 

analogs : 0 

cases_exampl es:  0 

percep_speed:  4 

search_rec_inf o : 
id_obj__act_e  vents  : 
scan_display : 4 

acceleration:  0 

confinement:  3 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  2 

magnetism:  0 

noise:  3 

fatigue:  4 

>nental_  strain : 4 

stress:  4 

phys_strain : 0 

preciseness:  0 

cog^attent : 4 


3 


response_chaining : 
attention_span:  4 

sleep:  4 

schedule:  4 

boredom:  0 

general_health:  4 

age : 0 

gender : 0 

height:  0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 2 

articulation:  3 

kr_hp:  1 

computer:  1 

vehicles:  0 

weapon_systems : 0 

instruments:  3 

notation:  0 

test_equip:  0 

advises : 3 

answers:  0 

communicates:  4 

directs : 4 

indicates:  0 

informs:  4 

instructs:  0 

requests:  4 

transmits:  3 

supervises:  4 

negotiates:  0 

express_movement : 0 

interp_movement : 0 

namel:  air_traffic 

name2 : program_mgmt 

eq_count:  57 

sim  count:  72 


statistical:  0 

spatial : 4 

temporal:  0 

analogical:  0 

case_based:  2 

model_based:  1 

mathematical:  0 

deductive : 0 

inductive:  0 

means_ends : 4 

sub_goals : 0 

gen_and_test : 0 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : 3 

acq_and_pres : 1 

form_proc_alg : 3 

decompose : 2 

recombine : 0 

generalize:  0 

specialize:  0 

ret_to_def : 3 

visual:  4 

verbal:  0 


auditory:  0 

kinesthetic : 0 

written:  3 

instrumentation:  4 

mrn_interf  ace : 1 

databases : 2 

historical:  4 

propadeutics : 1 

branching : 2 

dynamism:  3 

constraints : 2 

uncertainty:  2 

high_low_tech : 0 

quant__qual:  0 

complex_cont : 0 

compound:  0 

reflex:  1 

simple_disc : 2 

fine:  2 

gross:  0 

repetitive:  2 

categorize:  0 

calculate : 0 

code:  0 

computerize : 0 

interpolate:  3 

itemize:  1 

learn:  0 

tabulate:  0 

translate:  0 

analyze:  2 

deduce:  0 

induce:  0 

choose:  1 

.compare:  1 

compute : 0 

estimate:  2 

integrate : 0 

plan:  0 

supervise : 0 

monitor:  0 

interpret : 1 

facts:  2 

principles : 0 

procedures:  0 

analogs : 0 

cases_examples : 2 

percep_speed:  4 

search_rec_inf o : 4 

id_ob  j_act_e vents  : 4 

scan_display : 4 

acceleration:  0 

confinement : 3 

isolation:  0 

contaminants:  0 

electricity : 0 

lighting:  2 

magnetism:  0 

noise:  3 

fatigue:  4 

iental_strain : 4 

stress : 4 

phys_strain : 0 

preciseness:  0 

cog_attent : 4 

response_chaining:  3 


attention_span:  4 

sleep:  4 

schedule:  4 

boredom:  0 

general_health:  4 

age : 0 

gender:  0 

height:  0 

weight:  0 

edu_train_expert : 2 

intro_accuracy : 0 

prob_of_success : 2 

articulation:  3 

kr_hp : 1 

computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  4 

test_equip:  0 

advises:  3 

answers : 3 

communicates:  0 

directs:  2 

indicates:  0 

informs:  2 

instructs:  0 

requests:  1 

transmits:  1 

supervises:  1 

negotiates:  4 

express_movement : 0 

interp_movement : 0 

namel:  air_traffic 

name2 : drair_gen 

eq_count:  43 

sim  count:  66 


statistical:  1 

spatial:  4 

temporal:  2 

analogical:  1 

case_based:  0 

modeljbased:  1 

mathematical:  1 

deductive:  1 

inductive:  4 

means_ends : 4 

sub_goals:  2 

gen_and_test : 0 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : 0 

acq_and_pres : 2 

f orm_proc_alg : 3 

decompose : 3 

recombine : 3 

generalize:  3 

specialize:  0 

ret_to_def:  0 

visual:  4 

verbal:  0 

auditory:  0 


kinesthetic : 0 

written:  0 

instrumentation : 
mm_interf ace : 2 

databases : 4 

historical : 4 

propadeutics : 1 

branching : 4 

dynamism:  4 

constraints:  4 

uncertainty:  1 

high_low_tech : 1 

quant_qual:  0 

complex_cont : 0 

compound:  0 

reflex:  1 

simple_disc:  2 

fine:  2 

gross:  0 

repetitive : 2 

categorize:  1 

calculate:  0 

code : 0 

computerize:  0 

interpolate : 2 

itemize:  1 

learn:  1 

tabulate:  1 

translate : 2 

analyze:  4 

deduce:  1 

induce:  4 

choose:  4 

compare : 0 

compute : 0 

estimate:  1 

integrate:  4 

plan:  4 

supervise:  4 

monitor:  4 

interpret : 3 

facts:  0 

principles:  3 

procedures:  1 

analogs : 1 

cases_examples : 0 

percep_speed:  4 

search_rec_info : 
id_ob  j_act_events : 
scan_display : 4 

acceleration:  0 

confinement:  3 

isolation : 0 

contaminants:  0 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  3 

fatigue:  4 

mental_strain : 4 

stress:  4 

phys_strain:  0 

preciseness:  0 

cog_attent : 4 

response_chaining : 
attention_span:  4 


sleep:  4 

schedule:  4 

boredom:  0 

general_health:  4 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 3 

articulation:  1 

kr_hp : 1 

computer:  4 

vehicles:  0 

weapon_systems : 0 

instruments : 4 

notation:  3 

test_equip:  0 

advises:  3 

answers:  4 

communicates:  0 

directs:  4 

indicates:  0 

informs : 0 

instructs:  0 

requests:  2 

transmits:  1 

supervises:  4 

negotiates:  0 

express^movement : 0 

interp_movement : 0 

namel:  air_traffic 

name2 : cargo_loading 

eq_count : 48 

sim  count:  77 


statistical:  0 

spatial:  0 

temporal:  0 

analogical:  0 

casejbased:  1 

model  Jbased:  0 

mathematical:  4 

deductive:  1 

inductive:  0 

means_ends : 1 

sub_goals : 0 

gen_and_test : 0 

goal_vs_data : 1 

cover_and_dif f : 1 

prop_and_ref : 0 

acq_and__pres : 1 

form_proc_alg : 4 

decompose:  2 

recombine:  1 

generalize:  0 

specialize:  2 

ret_to_def:  2 

visual:  1 

verbal:  4 

auditory:  0 

kinesthetic:  0 


written:  3 

instrumentation : 
mm_interface : 2 

databases:  0 

historical : 0 

propadeutics : 1 

branching:  3 

dynamism:  3 

constraints : 3 

uncertainty : 1 

high_low_tech : 0 

quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  1 

simple_disc : • 2 
fine:  2 

gross:  0 

repetitive:  2 

categorize:  1 

calculate : 4 

code : 0 

computerize:  0 

interpolate : 1 

itemize:  1 

learn:  1 

tabulate:  0 

translate:  0 

analyze:  1 

deduce:  1 

induce:  0 

choose:  2 

compare : 2 

compute:  4 

estimate:  0 

integrate : 0 

plan:  0 

supervise : 1 

monitor:  0 

interpret : 1 

facts:  0 

principles:  1 

procedures:  2 

analogs : 0 

cases_examples : 1 

percep_speed:  4 

search_rec__info : 
id_obj_act__e  vents : 
scan_display : 4 

acceleration:  1 

confinement:  1 

isolation:  0 

contaminants:  0 

electricity : 0 

lighting:  2 

magnetism:  0 

noise:  0 

fatigue:  4 

mental__strain : 4 

stress:  4 

phys_strain:  1 

preciseness:  0 

cog_attent:  4 

response_chaining : 
attention_span : 4 

sleep:  4 


schedule:  4 

boredom:  2 

general_health:  2 

age : 0 

gender : 0 

height:  0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 3 

articulation:  2 

kr_hp : 1 

computer : 0 

vehicles:  2 

weapon_sy stems : 0 

instruments:  4 

notation:  2 

test_equip:  0 

advises : 3 

answers : 2 

communicates:  1 

directs:  1 

indicates : 0 

informs:  4 

instructs:  0 

requests:  2 

transmits:  3 

supervises:  0 

negotiates:  0 

express_movement : 2 

interp_movement : 0 

namel:  air_traffic 

name2 : sw_design 

eq_count : 50 

sim  count:  65 


statistical:  0 

spatial:  4 

temporal:  0 

analogical:  0 

case_based:  2 

model_based:  2 

mathematical:  1 

deductive:  0 

inductive:  2 

means_ends:  0 

sub_goals : 0 

gen_and_test : 4 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : 4 

acq_and_pres : 2 

form_proc_alg:  2 

decompose:  0 

recombine:  4 

generalize:  2 

specialize:  0 

ret_to_def : 1 

visual:  4 

verbal:  0 

auditory:  0 

kinesthetic:  0 

written:  3 


instrumentation : 4 

mm_interf ace : 2 
databases:  0 

historical:  0 

propadeutics : 0 

branching : 1 

dynamism:  4 

constraints:  3 

uncertainty : 0 

high_low_tech : 1 

quant_qual:  0 

complex_cont : 0 

compound:  0 

reflex:  1 

simple_disc : 2 

fine:  2 

gross : 0 

repetitive : 2 

categorize : 0 

calculate : 0 

code : 2 

computerize:  2 

interpolate : 3 

itemize:  2 

learn : 1 

tabulate:  0 

translate : 4 

analyze : 4 

deduce : 0 

induce:  2 

choose:  2 

compare : 0 

compute:  0 

estimate:  1 

integrate:  4 

plan:  1 

supervise:  3 

monitor:  4 

interpret:  2 

facts:  1 

principles:  0 

procedures:  2 

analogs : 2 

cases_examples : 1 

percep_speed:  4 

search_rec_info : 4 

id_ob  j_act_e vent  s : 4 

scan_display : 4 

acceleration:  0 

confinement:  3 

isolation : 0 

contaminants:  0 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  3 

fatigue:  4 

mental_strain : 4 

stress:  4 

phys_strain : 0 

preciseness:  0 

cog_attent : 4 

response_chaining : 3 

attention__span : 4 

sleep:  4 

schedule:  4 


boredom:  0 

general_health : 4 

age:  0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 3 

articulation:  0 

kr_hp : 1 

computer:  4 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  0 

test_equip:  0 

advises : 3 

answers:  0 

communicates:  1 

directs:  4 

indicates:  0 

informs : 4 

instructs : 0 

requests:  4 

transmits:  1 

supervises:  4 

negotiates:  4 

express_movement : 0 

interp_movement : 0 

namel:  air_traffic 

name2 : form_f ill_out 

eq_count:  39 

sim  count:  58 


statistical:  0 

spatial:  4 

temporal : 2 

analogical:  2 

case_based:  2 

model_based:  1 

mathematical:  2 

deductive:  0 

inductive:  2 

means_ends:  2 

sub_goals : 0 

gen_and_test : 3 

goal_vs_data : 1 

cover_and_diff : 2 

prop_and_ref : 2 

acq_and__pres : 2 

form_proc_alg:  4 

decompose:  1 

recombine:  3 

generalize:  3 

specialize:  0 

ret_to_def : 2 

visual:  4 

verbal:  1 

auditory:  0 

kinesthetic:  0 

written:  3 

instrumentation:  4 


mmJLnterf  ace : 2 

databases : 1 

historical : 2 

propadeutics : 1 

branching : 4 

dynamism:  4 

constraints : 2 

^ uncertainty:  1 

high_low_tech : 0 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  1 

simple_disc:  2 

fine:  2 

gross:  0 

repetitive : 2 

categorize:  1 

calculate:  2 

code : 0 

computerize:  0 

interpolate:  1 

itemize:  2 

learn:  2 

tabulate:  1 

translate : 0 

analyze:  2 

deduce:  0 

induce:  2 

choose:  0 

compare : 0 

compute:  2 

estimate:  2 

integrate:  1 

plan:  4 

supervise : 4 

monitor : 4 

interpret : 0 

facts:  2 

principles:  1 

procedures:  0 

analogs : 0 

cases_examples : 1 

percep_speed:  4 

search_rec_inf o : 4 

id_ob j_act_events : 4 

scan_display : 4 

acceleration : 0 

confinement : 3 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  2 

magnetism:  0 

noise:  3 

fatigue:  4 

mental_strain : 4 

stress:  4 

phys_strain:  0 

preciseness:  0 

cog_attent : 4 

response_chaining:  3 

attention_span : 4 

sleep:  4 

schedule:  4 

boredom:  0 


general_health : 4 

age : 0 

gender : 0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 3 

articulation:  3 

kr_hp:  1 

computer : 4 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  1 

test_equip:  0 

advises:  3 

answers:  3 

communicates:  1 

directs:  4 

indicates:  3 

informs : 2 

instructs:  0 

requests:  2 

transmits:  1 

supervises:  4 

negotiates:  4 

express_movement : 0 

interp__movement : 0 

namel : air_traf f ic 

name2 : language_train 

eq_count:  51 

sim  count:  68 


statistical:  0 

spatial:  4 

temporal:  4 

analogical:  3 

casejbased:  2 

model_based:  1 

mathematical:  0 

deductive : 2 

inductive:  2 

means_ends : 3 

sub_goals : 0 

gen_and_test : 1 

goal_vs_data : 1 

cover_and_dif f : 1 

prop_and_ref : 2 

acq_and_pres : 2 

form_proc_alg:  0 

decompose:  0 

recombine : 4 

generalize:  3 

specialize:  1 

ret_to_def:  0 

visual:  4 

verbal : 0 

auditory:  0 

kinesthetic:  0 

written:  3 

instrumentation:  0 

mm  interface:  2 


databases : 0 

historical : 0 

propadeutics : 1 

branching : 1 

dynamism:  0 

constraints:  1 

___  uncertainty:  2 

high_low_tech : 0 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  1 

simple_disc : 2 

fine:  2 

gross:  0 

repetitive : 2 

categorize : 0 

calculate : 0 

code:  0 

computerize:  0 

interpolate : 3 

itemize:  3 

learn:  1 

tabulate:  0 

translate : 4 

analyze:  3 

deduce:  2 

induce:  2 

choose:  3 

compare : 0 

compute:  0 

estimate:  4 

integrate:  0 

plan:  0 

^.-supervise:  0 

monitor:  0 

interpret:  3 

facts:  0 

principles:  1 

procedures : 2 

analogs : 3 

cases_examples : 2 

percep__speed:  4 

search_rec_inf o : 4 

id_ob  j_act_events : 4 

scan_display : 4 

acceleration:  0 

confinement:  3 

isolation : 0 

contaminants:  0 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  3 

fatigue:  4 

mental__strain : 4 

stress:  4 

phys_strain : 0 

preciseness:  0 

cog_attent : 4 

^esponse^chaining : 3 

-^attention_span : 4 

sleep:  4 

schedule:  4 

boredom:  0 

general_health : 4 


age : 0 

gender : 0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 4 

prob_of_success : 1 

articulation:  1 

kr_hp : 1 

computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  4 

test_equip:  0 

advises:  1 

answers:  4 

communicates:  0 

directs:  0 

indicates:  4 

informs : 0 

instructs:  4 

requests:  0 

transmits:  1 

supervises:  0 

negotiates:  0 

express_movement : 3 

interp_movement : 3 

namel : air_traffic 

name2 : leadership 

eq_count : 44 

sim  count:  67 


statistical:  0 

spatial:  4 

temporal:  2 

analogical:  1 

case_based:  2 

model_based:  1 

mathematical:  0 

deductive:  0 

inductive:  2 

means_ends:  4 

sub_goals : 2 

gen_and_test : 3 

goal_vs_data : 1 

cover_and_dif f : 1 

prop_and_ref : 3 

acq_and_pres : 1 

form_proc_alg:  3 

decompose:  1 

recombine:  0 

generalize:  1 

specialize:  0 

ret_to_def:  0 

visual:  4 

verbal:  0 

auditory:  0 

kinesthetic:  0 

written:  2 

instrumentation : 
mm_interf ace : 2 

databases : 2 


4 


historical:  3 

propadeutics : 1 

branching:  3 

dynamism:  4 

constraints : 4 

uncertainty : 0 

high_low_tech : 0 

quant_qual:  0 

complex_cont : 0 

compound:  0 

reflex:  1 

simple_disc : 2 

fine:  2 

gross:  0 

repetitive:  2 

categorize:  0 

calculate : 2 

code:  0 

computerize:  1 

interpolate:  3 

itemize:  0 

learn : 3 

tabulate:  0 

translate:  0 

analyze:  4 

deduce:  0 

induce:  1 

choose:  1 

compare : 0 

compute:  1 

estimate:  3 

integrate : 2 

plan:  1 

supervise:  0 

monitor:  4 

interpret : 1 

facts:  1 

principles:  0 

procedures:  0 

analogs:  1 

cases_examples : 1 

percep_ speed:  4 

search_rec_inf o : 4 
id_obj_act_e vents : 4 

scan_display : 4 

acceleration:  0 

confinement : 3 

isolation:  0 

contaminants : 0 

electricity : 0 

lighting:  2 

magnetism:  0 

noise:  3 

fatigue:  4 

mental_strain : 4 

stress:  4 

phys_strain:  0 

preciseness:  0 

cog_attent : 4 

response_chaining : 3 

attention_span : 4 

sleep:  4 

schedule:  4 

boredom:  0 

general_health : 4 

age : 0 


gender:  0 

height:  0 

weight:  0 

edu_train_expert : 2 

intro_accuracy : 4 

prob_of_success : 0 

articulation:  1 

kr_hp:  1 

computer:  2 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  2 

test_equip:  0 

advises : 1 

answers:  2 

communicates:  0 

directs : 2 

indicates:  2 

informs:  0 

instructs:  4 

requests:  1 

transmits:  0 

supervises:  1 

negotiates:  3 

express_movement : 3 

interp_movement : 3 

namel : air_traffic 

name2 : surgery 

eq_count:  59 

sim  count:  83 


statistical:  0 

spatial:  0 

temporal : 0 

analogical:  1 

case_based:  1 

model_based:  2 

mathematical:  0 

deductive : 3 

inductive:  2 

means_ends : 0 

sub_goals : 0 

gen_and_test : 0 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : 0 

acq_and_pres : 1 

f orm_proc_alg : 4 

decompose:  0 

recombine:  0 

generalize:  1 

specialize:  3 

ret_to_def:  1 

visual:  0 

verbal : 0 

auditory:  2 

kinesthetic:  4 

written:  3 

instrumentation:  0 

mm_interf ace : 1 

databases:  0 

historical:  4 


propadeutics : 0 

branching:  0 

dynamism:  2 

constraints:  2 

uncertainty:  1 

high_low_tech : 1 

quant_qual:  0 

complex__cont : 4 

compound:  2 

reflex:  1 

simple_disc:  2 

fine:  2 

gross:  4 

repetitive:  2 

categorize : 2 

calculate : 0 

code : 0 

computerize : 0 

interpolate:  3 

itemize:  3 

learn:  1 

tabulate:  0 

translate : 0 

analyze:  4 

deduce : 4 

induce:  2 

choose : 0 

compare : 0 

compute : 0 

estimate:  1 

integrate : 0 

plan:  0 

supervise : 0 

monitor:  0 

interpret : 3 

facts:  0 

principles : 1 

procedures : 0 

analogs : 0 

cases_examples : 1 

percep_speed:  2 

search_rec_inf o : 4 

id_ob j_act_events : 0 

scan_display : 4 

acceleration:  0 

confinement:  1 

isolation : 0 

contaminants:  1 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  3 

fatigue:  1 

mental_strain : 1 

stress:  0 

phys_strain:  3 

preciseness : 4 

cog_attent : 0 

response_chaining : 1 

attention__span : 0 

sleep:  2 

schedule:  0 

boredom:  1 

general__health : 
age : 0 

gender:  0 


4 


height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 1 

articulation:  2 

kr_hp : 1 

computer:  0 

vehicles : 0 

weapon_systems : 0 

instruments:  0 

notation:  0 

test_equip:  3 

advises:  0 

answers:  3 

communicates:  1 

directs : 0 

indicates:  3 

informs : 2 

instructs:  0 

requests : 2 

transmits:  3 

supervises:  0 

negotiates:  0 

express_movement : 2 

interp_movement : 0 

namel : air_traffic 

name2 : medical_diag 

eq_count:  47 

sim  count:  67 


statistical:  0 

spatial:  1 

temporal : 0 

analogical:  0 

case_based:  2 

model_based:  1 

mathematical:  0 

deductive : 4 

inductive : 2 

means_ends : 3 

sub_goals : 2 

gen_and_test : 0 

goal_vs_data : 0 

cover_and_dif f : 0 

prop_and_ref  :•  4 

acq_and_pres : 2 

form_proc_alg : 2 

decompose:  1 

recombine:  0 

generalize:  2 

specialize:  2 

ret_to_def:  2 

visual:  0 

verbal : 0 

auditory:  2 

kinesthetic:  1 

written:  3 

instrumentation:  0 

mm_interface:  2 

databases:  0 

historical:  4 

propadeutics : 0 


branching : 0 

dynamism:  4 

constraints:  3 

uncertainty:  2 

high_JLow_tech : 1 

quant_qual:  1 

_____  complex_cont : 0 

compound:  0 

reflex:  1 

simple__disc : 2 

fine:  1 

gross:  1 

repetitive : 2 

categorize:  2 

calculate:  0 

code:  0 

computerize:  0 

interpolate:  3 

itemize:  1 

learn:  1 

tabulate:  0 

translate:  3 

analyze:  4 

deduce : 4 

induce:  2 

choose:  0 

compare : 1 

compute : 0 

estimate:  2 

integrate : 0 

plan:  3 

supervise : 4 

monitor:  2 

interpret : 3 

facts:  0 

principles:  1 

procedures : 0 

analogs : 0 

cases_examples : 2 

percep_speed:  4 

search_rec_inf o : 4 

id_obj_act_e  vents : 4 

scan_display : 4 

acceleration:  0 

confinement : 3 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  3 

fatigue:  3 

mental__strain : 2 

stress:  1 

phys^strain:  0 

preciseness:  0 

cog_attent : 4 

response_chaining : 3 

attention_span : 4 

sleep:  3 

schedule:  2 

boredom:  1 

general_health:  4 

age : 0 

gender : 0 

height : 0 


weight : 0 

edu_train_expert : 0 

intro_accuracy : 3 

prob_of_success : 1 

articulation:  0 

kr_hp : 1 

computer : 0 

vehicles:  0 

weapon_systems : 0 

instruments:  2 

notation:  0 

test_equip:  2 

advises : 1 

answers : 4 

communicates:  0 

directs:  0 

indicates:  2 

informs:  0 

instructs:  1 

requests:  3 

transmits:  1 

supervises:  3 

negotiates:  0 

express_movement : 2 

interpjmovement : 3 

namel : air_traffic 

name2 : accounting 

eq_count:  48 
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gross:  0 

repetitive:  0 

categorize:  2 

calculate : 4 

code : 0 

computerize:  0 

interpolate:  1 

itemize:  0 

learn:  0 

tabulate:  1 

translate:  2 

analyze:  3 

deduce : 0 

induce:  4 

choose:  2 

compare : 2 

compute : 4 

estimate:  1 

integrate:  4 

plan:  4 

supervise:  3 

monitor:  4 

interpret:  4 

facts:  0 

principles:  4 


procedures:  1 

analogs:  1 

cases_examples : 1 

percep_speed:  0 

search__rec_inf o : 0 

id_ob  j_act_event  s : 0 

scan__display : 0 

acceleration:  1 

confinement : 2 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  3 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  1 

preciseness : 0 

cog_attent : 0 

response_chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  2 

general_health : 2 

age : 0 

gender : 0 

height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  3 

kr_  hp : 0 

computer:  4 

vehicles:  2 

weapon_systems : 0 

instruments : 0 

notation:  1 

test_equip:  0 

advises:  0 

answers:  2 

communicates:  1 

directs:  3 

indicates : 0 

informs : 4 

instructs : 0 

requests:  0 

transmits:  2 

supervises:  4 

negotiates : 0 

express_movement : 2 

interp_movement : 0 

namel:  drair_gen 

name2 : sw_design 

eq^count : 72 

sim  count:  95 


statistical:  1 

spatial : 0 

temporal:  2 


analogical : 1 

casejbased:  2 

model_based:  3 

mathematical:  0 

deductive : 1 

inductive:  2 

means_ends : 4 

sub_goals : 2 

gen_and__test : 4 

goal_vs_data:  2 

cover_and_dif f : 
prop_and_ref : 4 

acq_and_j>res : 0 

form_proc_alg:  1 

decompose : 3 

recombine:  1 

generalize:  1 

specialize:  0 

ret_to_def : 1 

visual:  0 

verbal:  0 

auditory:  0 

kinesthetic:  0 

written:  3 

instrumentation : 
mm_interf ace : 0 

databases:  4 

historical:  4 

propadeutics : 1 

branching:  3 

dynamism:  0 

constraints:  1 

uncertainty:  1 

high_low_tech : 2 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  1 

calculate:  0 

code:  2 

computerize:  2 

interpolate:  1 

itemize:  3 

learn:  0 

tabulate:  1 

translate:  2 

analyze:  0 

deduce:  1 

induce:  2 

choose:  2 

compare : 0 

compute : 0 

estimate:  0 

integrate:  0 

plan:  3 

supervise : 1 

monitor:  0 

interpret : 1 

facts:  1 

principles:  3 

procedures:  1 


analogs:  1 

cases_examples : 1 

percep_speed:  0 

search_rec_info : 0 

id_ob j_act_events : 0 

scan_display : 0 

acceleration:  0 

confinement : 0 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness:  0 

cog_attent : 0 

response_chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  1 

kr_hp : 0 

computer : 0 

vehicles:  0 

weapon_systems : 0 

instruments : 0 

notation:  3 

test_equip:  0 

advises : 0 

answers : 4 

communicates:  1 

directs:  0 

indicates:  0 

informs : 4 

instructs : 0 

requests:  2 

transmits : 0 

supervises:  0 

negotiates:  4 

express_movement : 0 

interp_movement : 0 

namel : drair_gen 

name2 : form_f ill_out 

eq_count:  74 

sim  count : 90 


statistical:  1 

spatial : 0 

temporal:  0 

analogical:  1 , 


case_based:  2 

model_based:  0 

mathematical:  1 

deductive:  1 

inductive:  2 

means_ends : 2 

sub_goals : 2 

gen_a  nd_t  e s t : 3 

goal_vs_data : 0 

cover_and_dif f : 
prop_and_ref : 2 

acq_and_j)res : 0 

form_proc_alg:  1 

decompose:  2 

recombine : 0 

generalize:  0 

specialize:  0 

ret_to_def : 2 

visual:  0 

verbal:  1 

auditory:  0 

kinesthetic:  0 

written:  3 

instrumentation : 
mm_interf ace : 0 

databases:  3 

historical:  2 

propadeutics : 2 

branching:  0 

dynamism:  0 

constraints:  2 

uncertainty:  2 

high_low_tech : 1 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross : 0 

repetitive:  0 

categorize:  2 

calculate:  2 

code : 0 

computerize:  0 

interpolate:  1 

itemize:  3 

learn:  1 

tabulate:  0 

translate:  2 

analyze:  2 

deduce : 1 

induce:  2 

choose:  4 

compare : 0 

compute : 2 

estimate:  1 

integrate:  3 

plan:  0 

supervise:  0 

monitor : 0 

interpret : 3 

facts:  2 

principles:  2 

procedures : 1 

analogs : 1 


cases_examples : 1 

percep_speed:  0 

search_rec_inf o : 0 

id_ob  j_ac t_e vent  s : 0 

scan_display : 0 

acceleration:  0 

confinement : 0 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain : 0 

preciseness:  0 

cog__attent:  0 

response_chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight:  0 

edu_train_expert : 0 

intro_accuracy : - 0 
prob_of_success : 0 

articulation:  4 

kr_hp : 0 

computer:  0 

vehicles:  0 

weapon_systems : 0 

instruments : 0 

notation:  2 

test_equip:  0 

advises : 0 

answers:  1 

communicates:  1 

directs:  0 

indicates:  3 

informs:  2 

instructs:  0 

requests:  0 

transmits : 2 

supervises:  0 

negotiates:  4 

express_movement : 0 

interp_movement : 0 

namel : drair_gen 

name2 : language_train 

eq_count : 65 

sim  count:  82 


statistical:  1 

spatial:  0 

temporal:  2 

analogical:  2 

case  based:  2 


model_based : 0 

mathematical:  1 

deductive:  1 

inductive : 2 

means_ends:  1 

sub_goals : 2 

gen_and_test : 1 

goal_vs_data : 2 

cover_and_dif f : 
prop_and_ref : 2 

acq_and_pres : 0 

f orm_proc_alg : 
decompose : 3 

recombine:  1 

generalize:  0 

specialize:  1 

ret_to_def:  0 

visual:  0 

verbal:  0 

auditory:  0 

kinesthetic:  0 

written:  3 

instrumentation : 
mm_interf ace : 0 

databases:  4 

historical:  4 

propadeutics : 2 

branching:  3 

dynamism:  4 

constraints:  3 

uncertainty:  1 

high_low_tech : 1 

quant_qual:  0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  1 

calculate:  0 

code : 0 

computerize : 0 

interpolate:  1 

itemize:  4 

learn:  0 

tabulate:  1 

translate:  2 

analyze:  1 

deduce:  1 

induce:  2 

choose:  1 

compare : 0 

compute : 0 

estimate:  3 

integrate:  4 

plan:  4 

supervise:  4 
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interpret : 0 

facts:  0 

principles:  4 

procedures:  1 

analogs:  2 

cases_examples_:  2 


percep_speed:  0 

search_rec_inf o : 0 

id_obj_act_e vents : 0 

scan_display : 0 

acceleration:  0 

confinement : 0 

isolation:  0 

contaminants : 0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness : 0 

cog_attent:  0 

response__chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health : 0 

age : 0 

gender : 0 

height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 4 

prob_of_success : 2 

articulation:  0 

kr_hp : 0 

computer:  2 

vehicles:  0 

weapon_systems : 0 

instruments:  0 

notation:  1 

test_equip:  0 

advises:  2 

answers : 0 

communicates:  0 

directs:  4 

indicates:  4 

informs:  0 

instructs : 4 

requests:  2 

transmits:  2 

supervises:  4 

negotiates:  0 

express_movement : 3 

interp__movement : 3 

namel:  drair_gen 
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eq_count:  67 

sim  count:  89 
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spatial:  0 

temporal:  0 

analogical : 0 

case_based:  2 

model  based:  0 


mathematical:  1 

deductive : 1 

inductive:  2 

means_ends : 0 

sub_goals : 0 

gen_and_test : 3 

goal_vs_data : 2 

cover_and_dif f : 0 

prop_and_ref : 3 

acq_and_j?res : 1 

f orm_proc_alg : 0 

decompose : 2 

recombine:  3 
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kinesthetic:  0 
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search_rec_inf o : 0 
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preciseness:  0 
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express_movement : 3 

interp_movement : 3 
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analogical:  0 

case_based:  1 
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mathematical:  1 
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inductive:  2 
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sub_goals : 2 

gen_and_test : 0 
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prop_and_ref : 0 
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decompose : 3 
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specialize : 3 
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kinesthetic:  4 
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instrumentation:  4 

mm_interf ace : 1 
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uncertainty : 0 
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fine:  4 
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calculate : 0 

code:  0 

computerize:  0 
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id_ob j_act_events : 
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contaminants:  1 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  0 

fatigue:  3 

mental_strain : 3 

stress:  4 

phys_strain:  3 

preciseness:  4 

cog_attent : 4 

response_chaining : 2 

attention_span : 4 

sleep:  2 

schedule:  4 

boredom:  1 

general_health:  0 

age : 0 

gender:  0 

height:  0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 4 

articulation:  3 

kr_hp:  2 

computer : 4 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  3 

test_equip:  3 

advises:  3 

answers : 1 

communicates : 1 

directs:  4 

indicates:  3 

informs : 2 
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transmits:  2 

supervises:  4 

negotiates:  0 

express_movement : 2 

interp_movement : 0 

namel : drair_gen 
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eq_count:  50 
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spatial:  3 

temporal : 2 

analogical:  1 
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model  Jbased:  2 

mathematical:  1 

deductive : 3 


inductive : 2 

means_ends : 1 

sub_goals : 0 

gen_and_test : 0 

goal_vs_data:  1 

cover_and_dif f : 1 

s ^prop_and_ref : 4 

acq_and_pres : 0 

f orm_proc_alg : 1 

decompose : 2 

recombine : 3 

generalize : 1 

specialize:  2 

ret_t°_def:  2 

visual:  4 

verbal : 0 

auditory:  2 

kinesthetic:  1 
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high_low_tech : 2 

quant _qual : 1 

complex_cont : 0 
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simple^ disc:  0 
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gross : 1 

repetitive : 0 

categorize:  3 

calculate : 0 

code:  0 

computerize:  0 

interpolate : 1 

itemize:  2 

learn:  2 

tabulate:  1 

translate:  1 

analyze:  0 

deduce : 3 

induce:  2 
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compute : 0 
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integrate:  4 

plan:  1 

supervise:  0 
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procedures : 1 
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cases_examples : 2 

percep_speed:  0 

search_rec__info : 0 
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scan_di splay:  0 

acceleration:  0 

confinement : 0 
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contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  0 

preciseness:  0 

cog_attent:  0 

response_chaining:  0 

attention_span : 0 

sleep:  1 

schedule:  2 

boredom:  1 

general_health : 0 

age : 0 

gender : 0 

height:  0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 3 

prob_of_success : 4 

articulation:  1 

kr_hp : 0 

computer:  4 

vehicles:  0 

weapon_systems : 0 

instruments : 2 

notation:  3 

test_equip:  2 

advises : 4 

answers : 0 

communicates:  0 

directs:  4 

indicates:  2 

informs:  0 

instructs:  1 

requests:  1 

transmits:  0 

supervises:  1 

negotiates:  0 

express_movement : 2 

interp_movement : 3 

namel : drair_gen 
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eq_count : 72 

sim  count:  101 


statistical:  1 

spatial : 0 

temporal:  1 

analogical:  1 

case_based:  1 

model_based:  0 

mathematical:  3 

deductive : 1 

inductive:  4 


means_ends : 0 

sub_goals : 1 

gen_and_test : 0 

goal_vs_data:  1 

cover_and_dif f : 0 

prop_and_ref : 0 

acq_and_j>res : 0 

f orm_proc_alg : 1 

decompose : 1 

recombine:  1 

generalize:  3 

specialize:  3 

ret_to_def:  0 

visual:  0 

verbal:  1 

auditory:  0 

kinesthetic : 0 

written:  3 

instrumentation : 
mm_interf ace : 4 

databases : 0 

historical : 0 

propadeutics : 2 

branching : 0 

dynamism:  1 

constraints:  1 

uncertainty:  1 

high_low_tech : 1 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc : 0 

fine:  0 

gross:  0 

repetitive : 0 

categorize:  1 

calculate : 4 

code:  0 

computerize:  4 

interpolate:  1 

itemize:  0 

learn:  1 

tabulate:  3 

translate : 1 

analyze:  4 

deduce : 1 

induce’:  4 

choose:  1 

compare : 0 

compute:  4 

estimate:  3 

integrate : 1 

plan:  0 

supervise : 0 

monitor:  0 

interpret : 2 

facts:  3 

principles : 2 

procedures:  1 

analogs : 1 

^ases_examples : 1 

percep_speed:  0 

search_rec_inf o : 
id_ob j_act_events : 
scan_display : 0 


acceleration:  0 

confinement : 0 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness : 4 

cog_attent:  0 

response_chaining : 0 

attention_span:  0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health:  0 

age : 0 

gender : 0 

height:  0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  4 

kr_hp : 0 

computer:  0 

vehicles:  0 

weapon_sys terns : 0 

instruments : 0 

notation:  2 

test_equip:  0 

advises:  0 

answers:  2 

communicates:  0 

directs:  0 

indicates:  0 

informs : 2 

instructs:  0 

requests:  1 

transmits:  1 

supervises:  0 

negotiates:  0 

express_movement : 0 

interp_movement : 0 

namel : drair_gen 

name2 : protocol_des 

eq_count:  69 

sim  count:  93 


statistical:  1 

spatial:  1 

temporal:  2 

analogical:  1 

case_based:  3 

model_based:  2 

mathematical:  1 

deductive:  1 

inductive:  1 

means  ends:  0 


sub_goals : 2 

gen_and_test : 4 

goal_vs_data : 2 

cover_and_dif f : 0 

prop_and_ref : 4 

acq_and_j?res : 0 

r'orm_proc_alg : 2 

decompose : 3 

recombine:  0 

generalize:  0 

specialize : 0 

ret_to_def : 1 

visual:  0 

verbal:  0 

auditory:  0 

kinesthetic : 0 

written:  2 

instrumentation : 2 

mm_interf ace : 0 

databases : 4 

historical:  0 

propadeutics : 4 

branching : 1 

dynamism:  0 

constraints:  0 

uncertainty : 2 

high_low_tech : 2 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

tross:  0 

repetitive:  0 

categorize:  3 

calculate:  3 

code : 0 

computerize:  0 

interpolate:  0 

itemize:  0 

learn:  0 

tabulate:  1 

translate:  2 

analyze:  4 

deduce:  1 

induce:  2 

choose:  4 

compare:  0 

compute:  3 

estimate:  1 

integrate:  1 

plan:  4 

supervise:  1 

monitor:  0 

interpret : 0 

facts:  1 

principles : 3 

procedures : 1 

analogs : 1 

ases_examples : 3 

^ercep_speed:  0 

search_rec_inf o : 0 

id_ob  j_act_events  : 0 

scan_display : 0 

acceleration : 0 


confinement : 0 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness:  0 

cog_attent : 0 

response_chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health:  0 

age : 0 

gender:  0 

height:  0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 4 

articulation:  1 

kr_hp:  0 

computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  1 

test_equip:  1 

advises : 0 

answers:  1 

communicates:  0 

directs : 3 

indicates : 0 

informs : 1 

instructs:  0 

requests:  0 

transmits:  1 

supervises:  3 

negotiates:  0 

express_movement : 0 

interp_movement : 0 

namel : cargo_loading 

name2 : sw_design 

eq_count : 59 

sim  count:  78 


statistical:  0 

spatial:  4 

temporal : 0 

analogical:  0 

case__based:  1 

model_based:  2 

mathematical:  3 

deductive:  1 

inductive:  2 

means_ends : 1 

sub_goals : 0 


gen_and_test : 4 

goal_vs_data:  2 

cover_and_dif f : 0 

prop_and_ref : 4 

acq_and_j?res : 1 

form_proc_alg:  2 

decompose : 2 

recombine:  3 

generalize:  2 

specialize : 2 

ret_to_def:  1 

visual:  3 

verbal:  4 

auditory:  0 

kinesthetic:  0 

written:  0 

instrumentation : 0 

mm_interf ace : 0 
databases:  0 

historical:  0 

propadeutics : 1 

branching : 2 

dynamism:  1 

constraints:  0 

uncertainty:  1 

high_low_tech : 1 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  1 

calculate : 4 

code : 2 

computerize:  2 

interpolate:  2 

itemize:  3 

learn:  0 

tabulate:  0 

translate : 4 

analyze:  3 

deduce:  1 

induce:  2 

choose:  0 

compare:  2 

compute:  4 

estimate:  1 

integrate : 4 

plan:  1 

supervise:  2 

monitor:  4 

interpret : 3 

facts:  1 

principles : 1 

procedures : 0 

analogs : ' 2 
cases_examples : 0 

percep_speed:  0 

search__rec_inf o : 0 

id_ob  j__act_events : 0 

scan_display : 0 

acceleration:  1 

confinement:  2 


isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  3 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  1 

preciseness:  0 

cog_attent : 0 

response_chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  2 

general_health:  2 

age : 0 

gender : 0 

height:  0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  2 

kr_hp : 0 

computer : 4 

vehicles:  2 

weapon_sys terns : 0 
instruments:  0 

notation:  2 

test_equip:  0 

advises : 0 

answers : 2 

communicates:  0 

directs : 3 

indicates:  0 

informs : 0 

instructs:  0 

requests:  2 

transmits:  2 

supervises:  4 

negotiates:  4 

express_movement : 2 

interp_movement : 0 

namel : cargo_loading 

name2 : f orm_f ill_out 

eq_count:  61 

sim  count:  83 


statistical:  0 

spatial:  4 

temporal : 2 

analogical:  2 

case_based:  1 

model_based : 1 

mathematical:  2 

deductive:  1 

inductive:  2 

means_ends : 1 

sub_goals : 0 

gen_and_test : ■ 3 


goal_vs__data : 0 

cover_and_dif f : 3 

prop_and_ref : 2 

acq_and_j?res : 1 

formj?roc_alg : 0 

decompose : 1 

recombine:  2 

generalize : 3 

specialize:  2 

ret_to_def:  0 

visual:  3 

verbal:  3 

auditory:  0 

kinesthetic : 0 

written:  0 

instrumentation:  0 

mm_interf ace : 0 

databases : 1 

historical:  2 

propadeutics : 0 

branching : 1 

dynamism:  1 

constraints:  1 

uncertainty:  0 

high_low_tech : 0 

quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc : 0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  0 

calculate : 2 

code:  0 

computerize : 0 

interpolate:  0 

itemize:  3 

learn : 1 

tabulate:  1 

translate : 0 

analyze:  1 

deduce:  1 

induce:  2 

choose:  2 

compare : 2 

compute : 2 

estimate:  2 • 

integrate : 1 

plan:  4 

supervise : 3 

monitor:  4 

interpret : 1 

facts:  2 

principles : 2 

procedures:  2 

analo.gs : 0 

cases_examples : 0 

percep_speed:  0 

search_rec_info : 0 

id_obj_act_e vents  : 
scan_display : 0 

acceleration:  1 

confinement : 2 

isolation : 0 


contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  3 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  1 

preciseness : 0 

cog_attent:  0 

response_chaining:  0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  2 

general_health:  2 

age : 0 

gender:  0 

height:  0 

weight:  0 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  1 

kr_hp : 0 

computer:  4 

vehicles:  2 

weapon_systems : 0 

instruments:  0 

notation:  1 

test_equip:  0 

advises : 0 

answers : 1 

communicates:  0 

directs:  3 

indicates:  3 

informs : 2 

instructs:  0 

requests:  0 

transmits : 4 

supervises:  4 

negotiates:  4 

express_movement : 2 

interp_movement : 0 

namel : cargo_loading 

name2 : language_train 

eq_count:  56 

sim  count:  73 


statistical:  0 

spatial:  4 

temporal:  4 

analogical:  3 

case_based:  1 

modeT_based:  1 

mathematical:  4 

deductive:  1 

inductive:  2 

means_ends : 2 

sub_goals : 0 

gen_and_test : 1 

goal_vs_data:  2 


cover_and_dif f : 
prop_and__ref : 2 

acq_and_j?res : 1 

form_proc_alg:  4 

decompose:  2 

recombine : 3 

generalize:  3 

specialize:  1 

ret_to__def : 2 

visual:  3 

verbal:  4 

auditory:  0 

kinesthetic:  0 

written:  0 

instrumentation : 
mm_interf ace : 0 

databases : 0 

historical : 0 

propadeutics : 0 

branching:  2 

dynamism:  3 

constraints:  2 

uncertainty:  3 

high_low_tech : 0 

quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  1 

calculate : 4 

code:  0 

computerize : 0 

interpolate:  2 

itemize:  4 

learn:  0 

tabulate:  0 

translate:  4 

analyze:  2 

deduce:  1 

induce:  2 

choose:  1 

compare : 2 

compute:  4 

estimate:  4 

integrate:  0 

plan:  0 

supervise : 1 

monitor:  0 

interpret : 4 

facts:  0 

principles:  0 

procedures:  0 

analogs : 3 

cases_examples : 
percep_speed:  0 

search__rec_info : 
id_ob  j_ac  t_e  vent  s 
scan_display : 0 

acceleration:  1 

confinement:  2 

isolation : 0 

contaminants:  0 


electricity:  0 

lighting:  0 

magnetism:  0 

noise:  3 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  1 

preciseness:  0 

cog_attent : 0 

response_chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  2 

general_health : 2 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 4 

prob_of_success : 2 

articulation:  3 

kr_hp ; 0 

computer : 2 

vehicles:  2 

weapon_sys terns : 0 
instruments:  0 

notation:  2 

test_equip:  0 

advises:  2 

answers : 2 

communicates:  1 

directs:  1 

indicates : 4 

informs : 4 

instructs:  4 

requests:  2 

transmits : 4 

supervises:  0 

negotiates:  0 

express_movement : 1 

interp_movement : 3 

namel : cargo_loading 

name2 : leadership 

eq_count:  50 

sim  count:  82 


statistical:  0 

spatial:  4 

temporal:  2 

analogical:  1 

case_based:  1 

model_based:  1 

mathematical:  4 

deductive : 1 

inductive:  2 

means_ends : 3 

sub_goals : 2 

gen_and_test : 3 

goal_vs_data:  2 

cover  and  diff:  0 


prop_and_ref : 3 

acq_and_j)res : 0 

f orm_proc_alg : 1 

decompose:  1 

recombine : 1 

generalize : 1 

specialize:  2 

ret_t°_def:  2 

visual:  3 

verbal:  4 

auditory:  0 

kinesthetic : 0 

written:  1 

instrumentation : 
mm_interf  ace ; 0 

databases : 2 

historical:  3 

propadeutics : 0 

branching:  0 

dynamism:  1 

constraints:  1 

uncertainty : 1 

high_low_tech : 0 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  1 

calculate : 2 

code : 0 

computerize:  1 

interpolate : 2 

itemize:  1 

learn:  2 

tabulate:  0 

translate : 0 

analyze:  3 

deduce : 1 

induce : 1 

choose:  1 

compare : 2 

compute:  3 

estimate:  3 

integrate:  2 

plan:  1 

supervise : 1 

monitor:  4 

interpret : 0 

facts:  1 

principles : 1 

procedures:  2 

analogs : 1 

cases_examples : 
percep_speed:  0 

search_rec_info : 
id__ob  j_act_event  s 
scan_display : 0 

.acceleration:  1 

confinement : 2 

isolation:  0 

contaminants:  0 

electricity:  0 


lighting:  0 

magnetism:  0 

noise:  3 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  1 

preciseness:  0 

cog_attent : 0 

response_chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  2 

general_health : 2 

age : 0 

gender:  0 

height:  0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 4 

prob_of_success : 3 

articulation:  3 

kr_hp : 0 

computer:  2 

vehicles:  2 

weapon_systems : 0 

instruments:  0 

notation:  0 

test_equip:  0 

advises : 4 

answers:  0 

communicates : 1 

directs:  1 

indicates:  2 

informs : 4 

instructs:  4 

requests:  1 

transmits : 3 

supervises:  1 

negotiates:  3 

express_movement : 1 

interp_movement : 3 

namel:  cargo_loading 

name2 : surgery 

eq_count:  45 

sim  count:  67 


statistical:  0 

spatial:  0 

temporal : 0 

analogical:  1 

case_based:  0 

model_based:  2 

mathematical:  4 

deductive:  2 

inductive:  2 

means_ends : 1 

sub_goals : 0 

gen_a  nd__t  e s t : 0 
goal_vs_data:  2 

cover_and_dif f : 2 

prop_and_ref : 0 


acq_and_jpres : 0 

formj?roc_alg:  0 

decompose : 2 

recombine:  1 

generalize:  1 

specialize:  1 

ret_to_def : 1 

visual;  1 
verbal:  4 

auditory:  2 

kinesthetic : 4 

written:  0 

instrumentation : 
mm_interf ace : 1 

databases:  0 

historical:  4 

propadeutics : 1 

branching : 3 

dynamism:  1 

constraints:  1 

uncertainty:  2 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 4 

compound:  2 

reflex:  0 

simple_disc : 4 

fine:  4 

gross : 4 

repetitive : 4 

categorize : 1 

calculate:  4 

code : 0 

computerize:  0 

interpolate : 2 

itemize:  4 

learn:  2 

tabulate:  0 

translate:  0 

analyze:  3 

deduce : 3 

induce:  2 

choose:  2 

compare : 2 

compute : 4 

estimate:  1 

integrate : 0 

plan:  0 

supervise : 1 

monitor : 0 

interpret : 4 

facts:  0 

principles : 0 

procedures:  2 

analogs : 0 

cases_examples : 0 

percep_speed:  2 

search_rec_inf o : 
id_ob j_act_events : 
scan_display : 0 

acceleration : 1 

confinement:  0 

isolation : 0 

contaminants:  1 

electricity : 0 

lighting:  2 


magnetism:  0 

noise:  3 

fatigue:  3 

mental_strain : 3 

stress:  4 

phys_strain:  2 

preciseness:  4 

cog_attent : 4 

response_chaining : 2 

attention_span : 4 

sleep:  2 

schedule:  4 

boredom:  1 

general_health : 2 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 4 

articulation:  0 

kr_hp : 2 

computer : 0 

vehicles:  2 

weapon_systems : 0 

instruments:  4 

notation:  2 

test_equip:  3 

advises:  3 

answers:  1 

communicates:  0 

directs:  1 

indicates:  3 

informs : 2 

instructs:  0 

requests:  0 

transmits:  0 

supervises:  0 

negotiates:  0 

express_movement : 0 

interp_movement : 0 

namel:  cargo_loading 

name2 : medical_diag 

eq_count:  45 

sim  count:  73 


statistical:  0 

spatial:  1 

temporal : 0 

analogical:  0 

case_based:  1 

modeljbased:  1 

mathematical:  4 

deductive:  3 

inductive:  2 

means_ends : 2 

sub_goals : 2 

gen_and_test : 0 

goal_vs_data : 1 

cover_and_dif f : 1 

prop_and_ref : 4 

acq_and_pres : 1 


form_proc_alg : 2 

decompose : 1 

recombine:  1 

generalize:  2 

specialize:  0 

ret_to_def : 0 

visual:  1 

verbal:  4 

auditory:  2 

kinesthetic : 1 

written:  0 

instrumentation : 
mm_interf ace : 0 

databases:  0 

historical : 4 

propadeutics : 1 

branching:  3 

dynamism:  1 

constraints:  0 

uncertainty : 3 

high_low_tech : 1 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple__disc : 0 

fine:  1 

gross:  1 

repetitive:  0 

categorize:  1 

calculate : 4 

code:  0 

computerize:  0 

interpolate:  2 

itemize:  2 

learn : 2 

tabulate:  0 

translate : 3 

analyze:  3 

deduce : 3 

induce:  2 

choose:  2 

compare : 1 

compute : 4 

estimate:  2 

integrate : 0 

plan:  3 

supervise:  3 

monitor:  2 

interpret : 4 

facts : 0 

principles:  0 

procedures:  2 

analogs : 0 

cases_examples : 
percep_speed:  0 

search_rec_inf o : 
id_obj_act_e  vents 
scan_display : 0 

acceleration:  1 

confinement:  2 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 


noise:  3 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  1 

preciseness:  0 

cog_attent:  0 

response_chaining : 0 

attention_span : 0 

sleep:  1 

schedule:  2 

boredom : 1 

general_health : 2 

age : 0 

gender : 0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 3 

prob_of_success : 4 

articulation:  2 

kr_hp : 0 

computer:  0 

vehicles:  2 

weapon_systems : 0 

instruments:  2 

notation:  2 

test_equip:  2 

advises:  4 

answers : 2 

communicates:  1 

directs:  1 

indicates : 2 

informs : 4 

instructs:  1 

requests:  1 

transmits:  2 

supervises:  3 

negotiates:  0 

express_movement : 0 

interp_movement : 3 

namel:  cargo_loading 

name2 : accounting 

eq_count:  66 

sim  count:  89 


statistical:  0 

spatial:  4 

temporal:  1 

analogical:  0 

case_based:  0 

model  Jbased : 1 

mathematical:  0 

deductive:  1 

inductive : 0 

means_ends : 3 

sub_goals : 1 

gen_and_test : 0 

goal_vs_data : 1 

cover_and_dif f : 0 

prop_and_ref : 0 

acq_and_pres : 1 

form_proc_alg : , 0 


decompose:  0 

recombine:  1 

generalize:  0 

specialize:  1 

ret_to_def : 2 

visual:  3 

verbal:  3 

auditory:  0 

kinesthetic : 0 

written:  0 

instrumentation : 0 

mm_interf ace : 4 
databases:  4 

historical : 4 

propadeutics : 0 

branching:  1 

dynamism:  0 

constraints:  0 

uncertainty:  1 

high_low_tech : 0 

quant_qual:  2 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  1 

calculate : 0 

code:  0 

computerize : 4 

interpolate:  2 

itemize:  0 

learn:  1 

tabulate:  4 

translate ; 1 

analyze:  1 

deduce : 1 

induce:  0 

choose:  1 

compare : 2 

compute:  0 

estimate:  4 

integrate : 3 

plan : 4 

supervise : 3 

monitor:  4 

interpret : 2 

facts:  3 

principles:  2 

procedures:  2 

analogs : 0 

cases_examples : 0 

percep__speed:  0 

search_rec_info : 0 

id__obj__act_e  vents  : 0 

scan_display : 0 

acceleration:  1 

confinement:  2 

isolation : 0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  3 


0 


fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  1 

preciseness:  4 

cog_attent:  0 

response_chaining : 
attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  2 

general_health : 2 

age : 0 

gender : 0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  1 

kr_hp:  0 

computer : 4 

vehicles:  2 

weapon_sy stems : 0 

instruments:  0 

notation:  1 

test_equip:  0 

advises:  0 

answers : 0 

communicates:  1 

directs : 3 

indicates : 0 

informs : 2 

instructs:  0 

requests:  1 

transmits:  3 

supervises:  4 

negotiates:  0 

express_movement : 2 

interp_movement : 0 

namel : cargo_loading 

name2 : protocol_des 

eq_count:  57 

sim  count:  80 


statistical : . 2 
spatial:  3 

temporal:  0 

analogical:  0 

case_based:  2 

model_based:  1 

mathematical:  2 

deductive : 1 

inductive:  3 

means_ends : 3 

sub_goals : 0 

gen_and_test : 4 

goal_vs_data : 2 

cover_and_dif f : 0 

prop_and_ref : 4 

acq_and_pres : 1 

form_proc_alg : 3 

decompose : 2 


recombine:  2 

generalize:  3 

specialize : 2 

ret_to_def : 1 

visual:  3 

verbal:  4 

auditory:  0 

kinesthetic : 0 

written:  1 

instrumentation:  2 

mm_interf ace : 0 

databases : 0 

historical : 4 

propadeutics : 2 

branching:  0 

dynamism:  1 

constraints : 1 

uncertainty : 0 

high_ low_tech : 1 

quant_ qual:  0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc : 0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  1 

calculate : 1 

code:  0 

computerize:  0 

interpolate:  1 

itemize:  0 

learn : 0 

, tabulate : 0 

translate:  4 

analyze:  1 

deduce : 1 

induce : 2 

choose:  2 

compare : 2 

compute:  1 

estimate:  2 

integrate:  3 

plan:  0 

supervise:  2 

monitor:  4 

interpret:  4 

facts:  1 

principles : 1 

procedures : 2 

analogs:  0 

cases_examples : 2 

percep__speed:  0 

search_rec_inf o : 0 
id__ob  j_act_events : 
scan_display : 0 

acceleration:  1 

confinement : 2 

isolation : 0 

contaminant  s : 0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  3 

fatigue:  0 


0 


mental_strain : 0 

stress:  0 

phys_strain:  1 

preciseness:  0 

cog_attent:  0 

response_chaining : 
attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  2 

general_health:  2 

age : 0 

gender : 0 

height:  0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 4 

articulation:  2 

kr_hp : 0 

computer:  2 

vehicles:  2 

weapon_systems : 0 

instruments:  4 

notation:  2 

test_equip:  1 

advises:  0 

answers : 1 

communicates:  1 

directs : 0 

indicates : 0 

informs : 3 

instructs : 0 

requests:  0 

transmits:  3 

supervises:  1 

negotiates:  0 

express_movement : 2 

interp_movement : 0 

namel : sw_design 

name2 : form_f ill_out 

eq_count : 73  ~ 

sim  count:  94 


statistical:  0 

spatial:  0 

temporal:  2 

analogical:  2 

case_based:  0 

model_based:  3 

mathematical:  1 

deductive:  0 

inductive:  0 

means_ends : 2 

sub_goals : 0 

gen_and_test : 1 

goal_vs_data:  2 

cover_and_dif f : 3 

prop_and_ref : 2 

acq_and_j?res : 0 

form_proc_alg:  2 

decompose : 1 

recombine:  1 


generalize:  1 

specialize:  0 

ret_to_def : 1 

visual:  0 

verbal : 1 

auditory:  0 

.kinesthetic : 0 

written:  0 

instrumentation : 0 

mm_interf ace : 0 

databases : 1 

historical : 2 

propadeutics : 1 

branching:  3 

dynamism:  0 

constraints:  1 

uncertainty:  1 

high_low_tech : 1 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc : 0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  1 

calculate : 2 

code : 2 

computerize : 2 

interpolate:  2 

itemize:  0 

learn:  1 

tabulate:  1 

translate:  4 

analyze:  2 

deduce:  0 

induce : 0 

choose:  2 

compare : 0 

compute:  2 

estimate:  1 

integrate : 3 

plan:  3 

supervise:  1 

monitor:  0 

interpret : 2 

facts : 1 

principles:  1 

procedures:  2 

analogs : 2 

cases_examples : 0 

percep__speed:  0 

search_rec_ inf o : 0 

id__obj__act_e  vents : 
scan_display : 0 

acceleration:  0 

confinement:  0 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental  strain:  0 


0 


stress:  0 

phys_strain:  0 

preciseness:  0 

cog_attent : 0 

response_chaining : 
attention_span:  0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health : 0 

age : 0 

gender : 0 

height:  0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  3 

kr_hp : 0 

computer:  0 

vehicles:  0 

weapon_systems : 0 

instruments:  0 

notation:  1 

test_equip:  0 

advises:  0 

answers : 3 

communicates:  0 

directs:  0 

indicates : 3 

informs : 2 

instructs:  0 

requests:  2 

transmits : 2 

supervises:  0 

negotiates:  0 

express_movement : 0 

interp_movement : 0 

namel : sw_design 

name2 : language_train 

eq_count:  69 

sim  count:  87 


statistical:  0 

spatial:  0 

temporal:  4 

analogical:  3 

case_based:  0 

model_based:  3 

mathematical:  1 

deductive:  2 

inductive : 0 

means_ends : 3 

sub_goals : 0 

gen_a  nd_t  e s t : 3 

goal_vs_data : 0 

cover_and_dif f : 2 
orop_and_ref : 2 

acq_and_j>res : 0 

form_proc_alg:  2 

decompose : 0 

recombine : 0 

generalize:  1 


specialize:  1 

ret_to_def : 1 

visual:  0 

verbal:  0 

auditory:  0 

kinesthetic:  0 

written:  0 

instrumentation : 4 

mm_interf ace : 0 

databases : 0 

historical:  0 

propadeutics : 1 

branching : 0 

dynamism:  4 

constraints:  2 

uncertainty:  2 

high_low_tech : 1 

quant_qual:  0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross : 0 

repetitive:  0 

categorize:  0 

calculate:  0 

code : 2 

computerize:  2 

interpolate : 0 

itemize:  1 

learn : 0 

tabulate:  0 

translate:  0 

.analyze:  1 

deduce : 2 

induce:  0 

choose:  1 

compare : 0 

compute : 0 

estimate:  3 

integrate : 4 

plan:  1 

supervise:  3 

monitor : 4 

interpret : 1 

facts:  1 

principles:  1 

procedures:  0 

analogs : 1 

cases_examples : 1 

percep_speed:  0 

search_rec_inf o : 0 

id_ob j_act_events : 0 

scan_display : 0 

acceleration:  0 

confinement : 0 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

.lagnetism:  0 

"noise : 0 

fatigue:  0 

mental_strain : 0 

stress:  0 


0 


phys_strain:  0 

preciseness:  0 

cog_attent:  0 

response_chaining : 
attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health : 0 

age : 0 

gender:  0 

height:  0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 4 

prob_of_success : 2 

articulation:  1 

kr_hp : 0 

computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments:  0 

notation:  4 

test_equip:  0 

advises : 2 

answers:  4 

communicates:  1 

directs:  4 

indicates:  4 

informs : 4 

instructs:  4 

requests:  4 

transmits : 2 

supervises:  4 

negotiates:  4 

express_movement : 3 

interp_movement : 3 

namel : sw_design 

name2 : leadership 

eq_count:  68 

sim  count:  90 


statistical:  0 

spatial:  0 

temporal:  2 

analogical:  1 

case_based:  0 

model_based : 3 

mathematical:  1 

deductive:  0 

inductive:  0 

means_ends : 4 

sub_goals : 2 

gen_and_test : 1 

goal_vs_data : 0 

cover_and_dif f : 0 

prop_and_ref : 1 

acq_and_pres : 1 

form_proc_alg:  1 

decompose:  1 

recombine:  4 

generalize:  1 

specialize:  0 


ret_to_def : 1 

visual:  0 

verbal:  0 

auditory:  0 

kinesthetic : 0 

written:  1 

Instrumentation : 0 

mm_interf ace : 0 

databases : 2 

historical:  3 

propadeutics : 1 

branching : 2 

dynamism:  0 

constraints:  1 

uncertainty:  0 

high_low_tech : 1 

quant__qual : 0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive : 0 

categorize:  0 

calculate:  2 

code:  2 

computerize:  1 

interpolate:  0 

itemize:  2 

learn:  2 

tabulate:  0 

translate:  4 

analyze:  0 

.deduce : 0 

induce:  1 

choose:  1 

compare : 0 

compute : 1 

estimate:  2 

integrate : 2 

plan:  0 

supervise:  3 

monitor:  0 

interpret : 3 

facts:  0 

principles:  0 

procedures:  2 

analogs : 1 

cases_examples : 0 

percep_speed:  0 

search_rec_info : 0 

id_obj_act_e  vents : 
scan_display : 0 

acceleration:  0 

confinement : 0 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 

_noise:  0 

fatigue:  0 

mental_strain : 
stress:  0 

phys_strain:  0, 


0 


0 


preciseness:  0 

cog_attent : 0 

response_chaining : 
attention_span:  0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health:  0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 2 

intro_accuracy : 4 

prob_of_success : 3 

articulation:  1 

kr_hp:  0 

computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments:  0 

notation:  2 

test_equip:  0 

advises:  4 

answers:  2 

communicates:  1 

directs:  2 

indicates : 2 

informs : 4 

instructs : 4 

requests:  3 

transmits:  1 

supervises : 3 

negotiates:  1 

express_movement : 3 

interp__movement : 3 

name  1 : s w_de  sign 

name2 : surgery 

eq_count:  45 

sim  count:  61 


statistical:  0 

spatial:  4 

temporal:  0 

analogical:  1 

case_based:  1 

model_based:  0 

mathematical:  1 

deductive:  3 

inductive : 0 

means_ends : 0 

sub_goals : 0 

gen_and_test : 4 

goal_vs_data:  0 

cover_and_dif f : 2 

prop_and_ref : 4 

acq_and_pres : 1 

f orm_proc_alg : 2 

decompose:  0 

recombine : 4 

generalize:  1 

specialize:  3 

ret  to  def:  0 


visual:  4 

verbal : 0 

auditory:  2 

kinesthetic : 4 

written:  0 

instrumentation:  4 

mm__interf  ace : 1 

databases : 0 

historical:  4 

propadeutics : 0 

branching:  1 

dynamism:  2 

constraints : 1 

uncertainty:  1 

high_low_tech : 0 

quant_qual:  0 

complex_cont : 4 

compound:  2 

reflex:  0 

simple_disc : 4 

fine:  4 

gross:  4 

repetitive:  4 

categorize : 2 

calculate:  0 

code:  2 

computerize:  2 

interpolate:  0 

itemize:  1 

learn:  2 

tabulate:  0 

translate : 4 

analyze:  0 

deduce:  4 

induce:  0 

choose:  2 

compare:  0 

compute : 0 

estimate:  0 

integrate:  4 

plan:  1 

supervise : 3 

monitor:  4 

interpret : 1 

facts:  1 

principles:  1 

procedures:  2 

analogs : 2 

cases_examples : 0 

percep_speed:  2 

search_rec_inf o : 0 

id_ob  j__act_e vents  : 
scan_display : 0 

acceleration:  0 

confinement : 2 

isolation : 0 

contaminant  s : 1 

electricity : 0 

lighting:  2 

magnetism:  0 

ioise:  0 

^fatigue:  3 

mental_strain : 
stress:  4 

phys_strain : 3 

preciseness:  4 


3 


cog_attent:  4 

response_chaining : 2 

attention_span:  4 

sleep:  2 

schedule:  4 

boredom:  1 

general_health:  0 

age : 0 

gender : 0 

height:  0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 4 

articulation:  2 

kr_hp : 2 

computer:  4 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  0 

test_equip:  3 

advises : 3 

answers : 3 

communicates:  0 

directs:  4 

indicates:  3 

informs:  2 

instructs:  0 

requests:  2 

transmits:  2 

supervises:  4 

negotiates:  4 

express_movement : 2 

interp_movement : 0 

namel : sw_design 

name2 : medical_diag 

eq_count : 57 

sim  count:  84 


statistical:  0 

spatial:  3 

temporal:  0 

analogical:  0 

case_based:  0 

model_based:  1 

mathematical:  1 

deductive:  4 

inductive:  0 

means_ends : 3 

sub_goals : 2 

gen_and_test : 4 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : 0 

acq_and_pres : 0 

form_proc_alg : 0 

decompose : 1 

recombine:  4 

generalize:  0 

specialize:  2 

ret_to_def:  1 

visual:  4 


verbal:  0 

auditory:  2 

kinesthetic : 1 

written:  0 

instrumentation : 
mm__interf  ace : 0 

databases : 0 

historical:  4 

propadeutics : 0 

branching:  1 

dynamism:  0 

constraints:  0 

uncertainty : 2 

high__low_tech : 0 

quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple^ disc:  0 

fine:  1 

gross:  1 

repetitive:  0 

categorize:  2 

calculate:  0 

code : 2 

computerize:  2 

interpolate:  0 

itemize:  1 

learn : 2 

tabulate:  0 

translate : 1 

analyze:  0 

deduce:  4 

induce:  0 

choose:  2 

compare : 1 

compute : 0 

estimate:  1 

integrate : 4 

plan:  2 

supervise:  1 

monitor:  2 

interpret : 1 

facts:  1 

principles : 1 

procedures:  2 

analogs : 2 

cases_examples : 
percep_speed:  0 

search_rec_inf o : 
id_obj_act_e vents 
scan_display:  0 

acceleration:  0 

confinement : 0 

isolation : 0 

contaminant  s : 0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  1 

/mental__strain : 2 

stress:  3 

phys_strain : 0 

preciseness:  0 

cog_attent : 0 


0 


response_chaining : 
attention_span:  0 

sleep:  1 

schedule:  2 

boredom:  1 

general_health:  0 

age : 0 

gender:.  0 
height : 0 

weight:  0 

edu_train_expert : 0 

intro_accuracy : 3 

prob_of_success : 4 

articulation:  0 

kr_hp:  0 

computer:  4 

vehicles:  0 

weapon_systems : 0 

instruments:  2 

notation:  0 

test_equip:  2 

advises : 4 

answers : 4 

communicates:  1 

directs:  4 

indicates : 2 

informs : 4 

instructs:  1 

requests:  1 

transmits:  0 

supervises:  1 

negotiates:  4 

express_movement : 2 

interp_movement : 3 

namel : sw_design 

name2 : accounting 

eq^count:  69 

sim  count:  89 


statistical:  0 

spatial:  0 

temporal:  1 

analogical:  0 

case_based:  1 

model_based:  3 

mathematical:  3 

deductive:  0 

inductive:  2 

means_ends : 4 

sub_goals : 1 

gen_and_test : 4 

goal_vs_data : 1 

cover_and_dif f : 0 

prop_and_ref : 4 

acq_and_pres : 0 

form_proc_alg:  2 

decompose : 2 

recombine:  2 

generalize:  2 

specialize:  3 

ret_to_def : 1 

visual:  0 

verbal:  1 


auditory:  0 

kinesthetic : 0 

written:  0 

instrumentation:  0 

mm_interf  ace : 4 

databases : 4 

-historical : 4 

v~'/propadeutics : 1 

branching : 3 

dynamism:  1 

constraints : 0 

uncertainty : 0 

high_low__tech : 1 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive : 0 

categorize:  0 

calculate : 4 

code : 2 

computerize:  2 

interpolate : 0 

itemize:  3 

learn : 1 

tabulate:  4 

translate:  3 

analyze:  4 

deduce : 0 

induce : 2 

choose : 1 

compare : 0 

compute : 4 

estimate:  3 

integrate:  1 

plan:  3 

supervise:  1 

monitor:  0 

interpret : 1 

facts:  2 

principles:  1 

procedures:  2 

analogs:  2 

cases_examples : 0 

percep_speed:  0 

search_rec_info : 0 

id_obj__act_e  vents : 0 

scan_display : 0 

acceleration:  0 

confinement:  0 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

cnental_strain : 0 

^—stress:  0 

^ phys_strain : 0 

preciseness:  4 

cog_attent : 0 

response_chaining : 0 


attention_span:  0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health:  0 

age : 0 

gender:  0 

height:  0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  3 

kr_hp : 0 

computer : 0 

vehicles:  0 

weapon_systems : 0 

instruments : 0 

notation:  1 

test_equip:  0 

advises : 0 

answers:  2 

communicates:  1 

directs:  0 

indicates:  0 

informs : 2 

instructs:  0 

requests:  1 

transmits:  1 

supervises:  0 

negotiates:  4 

express_movement : 0 

interp_movement : 0 

namel : sw_design 

name2 : protocol_des 

eq_count : 76 

sim  count:  96 


statistical:  2 

spatial:  1 

temporal:  0 

analogical:  0 

casejbased:  1 

model_based:  1 

mathematical:  1 

deductive : 0 

inductive:  1 

means_ends : 4 

sub_goals : 0 

gen_and_test : 0 

goal_vs_data:  0 

cover_and_dif f : 0 

prop_and_ref : 0 

acq_and_pres : 0 

form_proc_alg:  1 

decompose : 0 

recombine:  1 

Teneralize:  1 

specialize:  0 

ret_to_def:  0 

visual:  0 

verbal:  0 

auditory:  0 


kinesthetic : 0 

written:  1 

instrumentation : 
mm_ interface : 0 

databases : 0 

historical:  4 

propadeutics : 3 

branching:  2 

dynamism:  0 

constraints:'  1 
uncertainty : 1 

high_low_tech : 0 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc : 0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  2 

calculate:  3 

code:  2 

computerize:  2 

interpolate:  1 

itemize:  3 

learn:  0 

tabulate:  0 

translate : 0 

analyze:  4 

deduce : 0 

induce:  0 

choose:  2 

compare : 0 

compute : 3 

estimate:  1 

integrate : 1 

plan:  1 

supervise : 0 

monitor:  0 

interpret : 1 

facts:  0 

principles : 0 

procedures:  2 

analogs : 2 

cases_examples : 2 

percep_speed:  0 

search_rec_inf o : 
id_ob j_ act_ events : 
scan_display : 0 

acceleration:  0 

confinement : 0 

isolation : 0 

contaminant  s : 0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness:  0 

cog_attent : 0 

response^ chaining : 
attention_span : 0 


sleep:  0 

schedule:  0 

boredom:  0 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 4 

articulation:  0 

kr_hp : 0 

computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments : 4 

notation:  4 

test_equip:  1 

advises:  0 

answers : 3 

communicates:  1 

directs:  3 

indicates:  0 

informs:  3 

instructs:  0 

requests:  2 

transmits:  1 

supervises:  3 

negotiates:  4 

express_movement:  0 

interp_movement : 0 

namel:  form_f ill_out 

name2 : language_train 

eq_count : 63 

sim  count:  82 


statistical:  0 

spatial:  0 

temporal:  2 

analogical:  1 

case_based:  0 

model_based:  0 

mathematical:  2 

deductive:  2 

inductive:  Q 

means_ends : 1 

sub_goals : 0 

gen_and_test : 2 

goal_vs_data : 2 

cover_and_dif f : 1 

prop_and_ref : 0 

acq_and_pres : 0 

form_proc_alg:  4 

decompose : 1 

recoirtbine:  1 

generalize:  0 

specialize:  1 

ret_to_def:  2 

visual:  0 

verbal:  1 

auditory:  0 

kinesthetic:  0 


written:  0 

instrumentation : 
mm_interf ace : 0 

databases : 1 

historical : 2 

propadeutics : 0 

branching : 3 

dynamism:  4 

constraints:  1 

uncertainty : 3 

high_low__tech : 0 

quant_qual:  0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive:  0 

categorize : 1 

calculate : 2 

code : 0 

computerize : 0 

interpolate : 2 

itemize:  1 

learn:  1 

tabulate:  1 

translate : 4 

analyze:  1 

deduce:  2 

induce:  0 

choose:  3 

compare : 0 

compute:  2 

estimate:  2 

integrate : 1 

plan:  4 

supervise:  4 

monitor : 4 

interpret:  3 

facts:  2 

principles:  2 

procedures:  2 

analogs : 3 

cases_examples : 1 

percep_speed:  0 

search_rec_inf o : 
id_ob j_act_events : 
scan_display : 0 

acceleration:  0 

confinement:  0 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

. preciseness : 0 

cog__attent : 0 

response_chaining : 
attention_span:  0 

sleep:  0 


schedule:  0 

boredom:  0 

general_health:  0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 4 

prob_of_success : 2 

articulation:  4 

kr_hp : 0 

computer:  2 

vehicles:  0 

weapon_systems : 0 

instruments : 0 

notation:  3 

test_equip:  0 

advises:  2 

answers : 1 

communicates:  1 

directs:  4 

indicates:  1 

informs : 2 

instructs:  4 

requests:  2 

transmits : 0 

supervises:  4 

negotiates:  4 

express_movement : 3 

interp_movement : 3 

namel:  form_fill_out 

name2 : leadership 

eq_count:  67 

sim  count:  98 


statistical:  0 

spatial:  0 

temporal : 0 

analogical:  1 

case_based:  0 

model_based:  0 

mathematical:  2 

deductive : 0 

inductive : 0 

means_ends : 2 

sub_goals:  2 

gen_and_test : 0 

goal_vs_data : 2 

cover_and_dif f : 3 

prop=and_ref : 1 

acq_andjpres : 1 

form_proc_alg : 1 

decompose : 0 

recombine : 3 

generalize:  2 

specialize:  0 

ret_to_def : 2 

visual:  0 

verbal:  1 

auditory:  0 

kinesthetic:  0 

written:  1 


instrumentation : 
mm_interf ace : 0 

databases : 1 

historical:  1 

propadeutics : 0 

branching:  1 

dynamism:  0 

constraints : 2 

uncertainty : 1 

high_low__tech : 0 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc : 0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  1 

calculate : 0 

code : 0 

computerize:  1 

interpolate : 2 

itemize:  2 

learn:  1 

tabulate:  1 

translate : 0 

analyze:  2 

deduce : 0 

induce:  1 

choose:  1 

compare : 0 

compute:  1 

estimate:  1 

integrate:  1 

plan:  3 

supervise:  4 

monitor:  0 

interpret:  1 

facts:  1 

principles : 1 

procedures:  0 

analogs : 1 

cases_examples : 0 

percep_speed:  0 

search_rec_inf o : 
id_obj_act_e vents : 
scan_display : 0 

acceleration:  0 

confinement : 0 

isolation : 0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain : 0 

preciseness:  0 

cog_attent:  0 

response__chaining : 
attention_span:  0 

sleep:  0 

schedule:  0 


boredom:  0 

general_health:  0 

age : 0 

gender : 0 

height:  0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 4 

prob_of_success : 3 

articulation:  4 

kr_hp:  0 

computer:  2 

vehicles:  0 

weapon_systems : 0 

instruments:  0 

notation:  1 

test_equip:  0 

advises:  4 

answers : 1 

communicates:  1 

directs:  2 

indicates:  1 

informs : 2 

instructs:  4 

requests:  1 

transmits:  1 

supervises:  3 

negotiates:  1 

express_movement : 3 

interp_movement : 3 

namel : form_f ill_out 

name2 : surgery 

eq_count:  39 

sim  count:  60 


statistical:  0 

spatial:  4 

temporal:  2 

analogical:  1 

casejbased:  1 

model_based:  3 

mathematical:  2 

deductive : 3 

inductive : 0 

means_ends : 2 

sub_goals : 0 

gen_and_test : 3 

goal_vs_data:  2 

cover_and_dif f : 1 

prop_and_ref : 2 

acq_and_pres : 1 

form_proc_alg : 0 

decompose:  1 

recombine : 3 

generalize:  2 

specialize:  3 

ret_to_def : 1 

visual:  4 

/erbal : 1 

auditory:  2 

kinesthetic:  4 

written:  0 

instrumentation : 4 


mm_iute£face : 1 

databases:  1 

historical:  2 

propadeutics : 1 

branching:  4 

dynamism:  2 

constraints:  0 

'“'uncertainty:  2 

high_low_tech : 1 

quant_qual : 0 

complex_cont : 4 

compound:  2 

reflex:  0 

s impl e_di sc:  4 

fine:  4 

gross:  4 

repetitive:  4 

categorize:  1 

calculate:  2 

code : 0 

computerize:  0 

interpolate:  2 

itemize:  1 

learn:  3 

tabulate:  1 

translate:  0 

analyze:  2 

deduce : 4 

induce:  0 

choose:  0 

compare : 0 

compute : 2 

estimate:  1 

■integrate:  1 

plan:  4 

''"''"supervise:  4 

monitor:  4 

interpret : 3 

facts:  2 

principles:  2 

procedures:  0 

analogs:  0 

cases_examples : 0 

percep_speed:  2 

search_rec_inf o : 0 

id_ob  j_act_events : 4 

scan_display : 0 

acceleration:  0 

confinement : 2 

isolation:  0 

contaminants:  1 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  0 

fatigue:  3 

mental_strain : 3 

stress:  4 

phys_strain:  3 

preciseness:  4 

■"og_attent : 4 

\,esponse_chaining : 2 

" &ttention_span : 4 

sleep:  2 

schedule:  4 

boredom:  1 


general_health : 0 

age : 0 

gender : 0 

height:  0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 4 

articulation:  1 

kr_hp : 2 

computer:  4 

vehicles:  0 

weapon_systems : 0 

instruments : 4 

notation:  1 

test_equip:  3 

advises:  3 

answers:  0 

communicates:  0 

directs:  4 

indicates : 0 

informs : 0 

instructs:  0 

requests:  0 

transmits:  4 

supervises:  4 

negotiates:  4 

express_movement : 2 

interp_movement : 0 

namel : form_f ill_out 

name2 : medical_diag 

eq_count:  47 

sim  count:  77 


statistical:  0 

spatial : 3 

temporal : 2 

analogical:  2 

case_based:  0 

model_based:  2 

mathematical:  2 

deductive : 4 

inductive:  0 

means_ends : 1 

sub_goals : 2 

gen_and_test : 3 

goal_vs_data : 1 

cover_and_dif f : 2 

prop_and_ref : 2 

acq_and_pres : 0 

form_proc_alg:  2 

decompose:  0 

recombine : 3 

generalize:  1 

specialize:  2 

ret_to_def : 0 

visual:  4 

verbal:  1 

auditory:  2 

kinesthetic:  1 

written:  0 

instrumentation : 
mm  interface:  0 


4 


databases : 1 

historical:  2 

propadeutics : 1 

branching:  4 

dynamism:  0 

constraints : 1 

uncertainty : 3 

high_low_tech : 1 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  1 

gross:  1 

repetitive:  0 

categorize : 1 

calculate : 2 

code : 0 

computerize:  0 

interpolate:  2 

itemize:  1 

learn:  3 

tabulate:  1 

translate:  3 

analyze:  2 

deduce : 4 

induce : 0 

choose:  0 

compare:  1 

compute:  2 

estimate:  0 

integrate:  1 

plan:  1 

supervise:  0 

monitor:  2 

interpret : 3 

facts:  2 

principles : 2 

procedures : 0 

analogs:  0 

cases_examples : 1 

percep_speed:  0 

search_rec_inf o : 
id_obj_act_e vents : 
scan_display : 0 

acceleration:  0 

confinement:  0 

isolation : 0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain : 0 

preciseness:  0 

cog_attent : 0 

response_chaining : 
.attention_span : 0 

sleep:  1 

schedule:  2 

boredom:  1 

general_health : 0 


age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 3 

prob_of_success : 4 

articulation:  3 

kr_hp:  0 

computer : 4 

vehicles:  0 

weapon_systems : 0 

instruments : 2 

notation:  1 

test_equip:  2 

advises : 4 

answers : 1 

communicates:  1 

directs:  4 

indicates:  1 

informs:  2 

instructs:  1 

requests:  1 

transmits:  2 

supervises:  1 

negotiates:  4 

express_movement : 2 

interp_movement : 3 

namel:  form_f ill_out 

name2 : accounting 

eq_count:  76 

sim  count:  95 


statistical:  0 

spatial:  0 

temporal:  1 

analogical:  2 

case_based:  1 

model_based:  0 

mathematical:  2 

deductive:  0 

inductive:  2 

means_ends:  2 

sub_goals : 1 

gen_and_test : 3 

goal_vs_data:  1 

cover_and_dif f : 3 

prop_and_ref : 2 

acq_and_j?res : 0 

f orm_proc_alg : 0 

decompose : 1 

recombine:  1 

generalize:  3 

specialize:  3 

ret_to_def:  2 

visual:  0 

verbal:  0 

auditory:  0 

kinesthetic:  0 

written:  0 

instrumentation:  0 

mm_interf ace : 4 

databases : 3 


historical: . 2 
propadeut ics : 0 

branching : 0 

dynamism:  1 

constraints : 1 

uncertainty:  1 

jhigh_low_tech : 0 

quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc : 0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  .1 

calculate : 2 

code : 0 

computerize : 4 

interpolate:  2 

itemize:  3 

learn:  2 

tabulate:  3 

translate:  1 

analyze:  2 

deduce:  0 

induce:  2 

choose:  3 

compare : 0 

compute : 2 

estimate : 2 

integrate : 2 

plan:  0 

supervise:  0 

monitor : 0 

interpret : 1 

facts:  1 

principles : 0 

procedures : 0 

analogs:  0 

cases_examples : 0 

percep_speed:  0 

search_rec_inf o : 
id_ob j_act_events : 
scan_display : 0 

acceleration:  0 

confinement:  0 

isolation : 0 

contaminants:.  0 
electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain:  0 

stress:  0 

phys_strain:  0 

preciseness : 4 

cog_attent : 0 

response_chaining : 
attention_span : 0 

jleep:  0 

schedule:  0 

boredom:  0 

general_health : 
age : 0 


0 


gender:  0 

height : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  0 

kr_hp : 0 

computer:  0 

vehicles:  0 

weapon_systems : 0 

instruments:  0 

notation:  0 

test_equip:  0 

advises : 0 

answers : 1 

communicates:  1 

directs:  0 

indicates:  3 

informs : 0 

instructs:  0 

requests:  1 

transmits:  1 

supervises:  0 

negotiates:  4 

express_movement : 0 

interp_movement : 0 

namel:  f orm_f ill_out 

name 2 : protocol_des 

eq_count:  68 

sim  count:  94 


statistical:  2 

spatial:  1 

temporal:  2 

analogical : 2 

case_based:  1 

model_based:  2 

mathematical:  0 

deductive : 0 

inductive:  1 

means_ends : 2 

sub_goals : 0 

gen_and_test : 1 

goal_vs_data:  2 

cover_and__diff : 3 

prop_and_ref : 2 

acq_and_pres : 0 

form_proc_alg:  3 

decompose:  1 

recombine:  0 

generalize:  0 

specialize:  0 

ret_to_def:  1 

visual:  0 

verbal : 1 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  2 

mm_interface:  0 

databases:  1 

historical:  2 


propadeutics s 2 
branching:  1 

dynamism:  0 

constraints : 2 

uncertainty:  0 

high_low_tech : 1 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross : 0 

repetitive:  0 

categorize:  1 

calculate:  1 

code:  0 

computerize:  0 

interpolate:  1 

itemize:  3 

learn:  1 

tabulate:  1 

translate:  4 

analyze:  2 

deduce : 0 

induce:  0 

choose:  0 

compare : 0 

compute : 1 

estimate:  0 

integrate:  2 

plan:  4 

supervise:  1 

monitor:  0 

^interpret : 3 

facts:  1 

principles:  1 

procedures:  0 

analogs : 0 

cases_examples : 2 

percep_speed:  0 

search_rec_info : 
id_obj_act_e vents : 
scan_display : 0 

acceleration:  0 

confinement : 0 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness : 0 

cog_attent:  0 

response_chaining : 
attention_span : 0 

sleep:  0 

Schedule:  0 

boredom:  0 

general_health : 
age : 0 

gender : 0 


0 


height : 0 

weight:  0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 4 

articulation:  3 

kr_hp : 0 

computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  3 

test_equip:  1 

advises:  0 

answers : 0 

communicates:  1 

directs:  3 

indicates:  3 

informs:  1 

instructs:  0 

requests:  0 

transmits:  1 

supervises:  3 

negotiates:  4 

express_movement : 0 

interp_movement : 0 

namel:  language_train 

name2 : leadership 

eq_count:  70 

sim  count:  90 


statistical:  0 

spatial:  0 

temporal : 2 

analogical:  2 

case_based:  0 

model_based:  0 

mathematical:  0 

deductive:  2 

inductive:  0 

means_ends : 1 

sub_goals : 2 

gen_and_test : 2 

goa l_vs_dat  a : 0 

cover_and_dif f : 2 

prop_and_ref : 1 

acq_and_pres : 1 

f orm_proc_alg : 3 

decompose : 1 

recombine:  4 

generalize:  2 

specialize:  1 

ret_to_def:  0 

visual:  0 

verbal : 0 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  4 

mm_interf ace : 0 

databases : 2 

historical:  3 

propadeutics : 0 


branching:  2 

dynamism:  4 

constraints:  3 

uncertainty:  2 

high_low_tech : 0 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  0 

calculate:  2 

code : 0 

computerize:  1 

interpolate:  0 

itemize:  3 

learn : 2 

tabulate:  0 

translate:  4 

analyze:  1 

deduce : 2 

induce:  1 

choose:  2 

compare : 0 

compute : 1 

estimate:  1 

integrate:  2 

plan:  1 

supervise : 0 

monitor:  4 

interpret:  4 

facts:  1 

principles:  1 

procedures:  2 

analogs : 2 

cases_examples : 1 

percep_speed:  0 

search_rec_inf o : 0 

id_ob j_act_events : 0 

scan_display : 0 

acceleration:  0 

confinement : 0 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness : 0 

cog_attent : 0 

response_chaining:  0 

attention_span : 0 

sleep:  0 

schedule:  0 

ioredom:  0 

general_health : 0 

age : 0 

gender:  0 

height : 0 


weight : 0 

edu_train_expert : 1 

intro_accuracy : 0 

prob_of_success : 1 

articulation:  0 

kr_hp : 0 

computer:  0 

vehicles:  0 

weapon_systems : 0 

instruments : 0 

notation:  2 

test_equip:  0 

advises:  2 

answers:  2 

communicates:  0 

directs:  2 

indicates:  2 

informs : 0 

instructs:  0 

requests:  1 

transmits:  1 

supervises:  1 

negotiates:  3 

express_movement : 0 

interp_movement : 0 

namel : language_train 

name2 : surgery 

eq_count : 46 

sim  count:  67 


statistical:  0 

spatial:  4 

temporal:  4 

analogical:  2 

case_based:  1 

model_based:  3 

mathematical:  0 

deductive : 1 

inductive : 0 

means_ends : 3 

sub_goals : 0 

gen_and_test : 1 

goal_vs_data : 0 

cover_and_dif f : 0 

prop_and_ref : 2 

acq_and_pres : 1 

form_proc_alg:  4 

decompose : 0 

recombine:  4 

generalize:  2 

specialize:  2 

ret_to_def:  1 

visual:  4 

verbal:  0 

auditory:  2 

kinesthetic:  4 

written:  0 

instrumentation:  0 

.nm_interf  ace : 1 

databases : 0 

historical:  4 

propadeutics : 1 

branching:  1 


dynamism:  2 

constraints : 1 

uncertainty : 1 

high_low_tech : 1 

quant_qual : 0 

complex__cont : 4 

compound:  2 

reflex:  0 

simple_disc:  4 

fine:  4 

gross:  4 

repetitive:  4 

categorize : 2 

calculate : 0 

code : 0 

computerize:  0 

interpolate:  0 

itemize:  0 

learn:  2 

tabulate:  0 

translate:  4 

analyze:  1 

deduce:  2 

induce:  0 

choose:  3 

compare : 0 

compute : 0 

estimate:  3 

integrate:  0 

plan:  0 

supervise : 0 

monitor:  0 

interpret : 0 

facts:  0 

principles:  0 

procedures:  2 

analogs : 3 

cases_examples : 1 

percep_speed:  2 

search_rec_info : 0 

id_ob  j_act_e vents  : 4 

scan_display : 0 

acceleration:  0 

confinement : 2 

isolation:  0 

contaminant  s : 1 

electricity : 0 

lighting:  2 

magnetism:  0 

noise:  0 

fatigue:  3 

mental_strain : 3 

stress:  4 

phys__strain : 3 

preciseness : 4 

cog_attent : 4 

response_chaining : 2 

attention_span : 4 

sleep:  2 

schedule:  4 

boredom:  1 

jeneral_health : 0 

age : 0 

gender:  0 

height : 0 

weight : 0 


edu_train_expert ; 1 

intro_accuracy : 4 

prob_of_success : 2 

articulation:  3 

kr_hp : 2 

computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  4 

test_equip:  3 

advises : 1 

answers : 1 

communicates:  1 

directs:  0 

indicates:  1 

informs : 2 

instructs:  4 

requests:  2 

transmits:  4 

supervises:  0 

negotiates:  0 

express_movement : 1 

interp_movement : 3 

namel:  language_train 

name2 : medical_diag 

eq_count:  60 

sim  count:  83 


constraints:  2 

uncertainty:  0 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc : 0 

fine:  1 

gross:  1 

repetitive:  0 

categorize:  2 

calculate : 0 

code:  0 

computerize:  0 

interpolate:  0 

itemize:  2 

learn:  2 

tabulate:  0 

translate : 1 

analyze:  1 

deduce : 2 

induce:  0 

choose:  3 

compare:  1 

compute:  0 

estimate:  2 

integrate:  0 

plan:  3 

supervise:  4 

monitor : 2 

interpret:  0 

facts:  0 

principles:  0 

procedures : 2 

analogs : 3 

cases_examples : 0 

percep_speed:  0 

search_rec_inf o : 
id_ob j_act_events : 
scan_display : 0 

acceleration:  0 

confinement:  0 

isolation : 0 

contaminant  s : 0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain : 0 

preciseness : 0 

cog^attent : 0 

response_chaining : 
attention_span : 0 

sleep:  1 

schedule:  2 

boredom:  1 

qeneral_health:  0 

>ge:  0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 


intro_accuracy : 1 

prob_of_success : 2 

articulation:  1 

kr_hp : 0 

computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments:  2 

notation:  4 

test_equip:  2 

advises : 2 

answers : 0 

communicates:  0 

directs:  0 

indicates:  2 

informs : 0 

instructs:  3 

requests:  3 

transmits : 2 

supervises:  3 

negotiates:  0 

express_movement : 1 

interp_movement : 0 

namel : language_train 

name2 : accounting 

eq_count:  58 

sim  count : 69 


uncertainty:  2 

high_low_tech : 0 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross:  0 

repetitive : 0 

categorize:  0 

calculate : 4 

code:  0 

computerize : 4 

interpolate : 0 

itemize:  4 

learn:  1 

tabulate:  4 

translate : 3 

analyze:  3 

deduce:  2 

induce:  2 

choose:  0 

compare : 0 

compute : 4 

estimate:  0 

integrate : 3 

plan:  4 

supervise : 4 

monitor:  4 

interpret : 2 

facts:  3 

principles:  2 

procedures:  2 

/analogs:  3 

cases_examples : 1 

percep_speed:  0 

search_rec_inf o : 
id_obj_act_e vents : 
scan_display : 0 

acceleration:  0 

confinement:  0 

isolation : 0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain : 0 

preciseness:  4 

cog_attent:  0 

response_chaining : 
attention_span : 0 

sleep:  0 

schedule : 0 

boredom:  0 

general_health : 0 

age:  0 

.gender : 0 

height : 0 

weight : 0 

edu_train_expert : 
intro_accuracy : 4 


2 


prob_of_success : 
articulation:  4 

kr_hp : 0 

computer:  2 

vehicles:  0 

weapon_sy stems : 0 

instruments:  0 

notation:  3 

test_equip:  0 

advises:  2 

answers : 2 

communicates:  0 

directs:  4 

indicates : 4 

informs : 2 

instructs:  4 

requests:  3 

transmits:  1 

supervises:  4 

negotiates:  0 

express_movement : 3 

interp_movement : 3 

namel : language_train 

name2 : protocol_des 

eq_count:  62 

sim  count:  85 


statistical:  2. 

spatial:  1 

temporal:  4 

analogical:  3 

case_based:  1 

model_based:  2 

mathematical : 2 

deductive : 2 

inductive:  1 

means_ends : 1 

sub_goals : 0 

gen_and_test : 3 

goal_vs_data:  0 

cover_and_dif f : 2 

prop_and_ref : 2 

acq_and_pres : 0 

form_proc_alg : 1 

decompose:  0 

recombine:  1 

generalize:  0 

specialize:  1 

ret_to_def:  1 

visual:  0 

verbal : 0 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  2 

mm^interf ace : 0 

databases : 0 

historical:  4 

propadeutics : 2 

branching:  2 

dynamism:  4 

constraints:  3 

uncertainty:  3 


high_low_tech : 1 

quant_qual:  1 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc : 0 

fine:  0 

gross:  0 

repetitive:  0 

categorize:  2 

calculate : 3 

code : 0 

computerize:  0 

interpolate:  1 

itemize:  4 

learn : 0 

tabulate:  0 

translate:  0 

analyze:  3 

deduce : 2 

induce:  0 

choose:  3 

compare : 0 

compute : 3 

estimate:  2 

integrate:  3 

plan:  0 

supervise:  3 

monitor : 4 

interpret : 0 

facts:  1 

principles:  1 

procedures:  2 

analogs : 3 

cases_examples : 1 

percep_speed:  0 

search_rec_inf o : 0 

id_ob j_act_events : 0 

scan_display : 0 

acceleration:  0 

confinement : 0 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness : 0 

cog_attent : 0 

response_chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health : 0 

age : 0 

gender : 0 

weight : 0 

weight : 0 

edu_train_expert : 1 

intro_accuracy : 4 

prob_of_success : 2 


articulation:  1 

kr_hp : 0 

computer : 0 

vehicles:  0 

weapon_systems : 0 

instruments : 4 

notation:  0 

test_equip:  1 

advises:  2 

answers : 1 

communicates:  0 

directs:  1 

indicates:  4 

informs : 1 

instructs:  4 

requests:  2 

transmits:  1 

supervises:  1 

negotiates:  0 

express_movement : 3 

interp_movement : 3 

namel : leadership 

name2 : surgery 

eq_count:  34 

sim  count:  62 


statistical:  0 

spatial:  4 

temporal : 2 

analogical:  0 

case_based:  1 

modeljbased:  3 

mathematical:  0 

deductive : 3 

inductive:  0 

means_ends : 4 

sub_goals : 2 

gen_and_test : 3 

goal_vs_data : 0 

cover_and_dif f : 2 

prop_and_ref : 3 

acq_and__pres : 0 

form_proc_alg:  1 

decompose : 1 

recombine:  0 

generalize:  0 

specialize:  3 

ret_to_def:  1 

visual:  4 

verbal:  0 

auditory:  2 

kinesthetic:  4 

written:  1 

instrumentation:  4 

mm_interf ace : 1 

databases : 2 

historical:  1 

propadeutics : 1 

branching:  3 

dynamism:  2 

constraints:  2 

uncertainty:  1 

high  low  tech : 1 


quant_qual : 0 

complex_cont : 4 

compound:  2 

reflex:  0 

simple_disc:  4 

fine:  4 

.gross:  4 

repetitive : 4 

categorize:  2 

calculate : 2 

code : 0 

computerize : 1 

interpolate : 0 

itemize:  3 

learn:  4 

tabulate:  0 

translate : 0 

analyze:  0 

deduce:  4 

induce:  1 

choose:  1 

compare : 0 

compute : 1 

estimate:  2 

integrate : 2 

plan:  1 

supervise:  0 

monitor:  4 

interpret : 4 

facts : 1 

principles:  1 

procedures : 0 

analogs : 1 

cases_examples : 0 

, per cep_speed : 2 

search_rec_inf o : 
id_ob  j_act_e vents : 
scan_display : 0 

acceleration:  0 

confinement : 2 

isolation : 0 

contaminants:  1 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  0 

fatigue:  3 

mental_strain : 3 

stress:  4 

phys_strain:  3 

preciseness : 4 

cog^attent : 4 

response_chaining : 
attention_span : 4 

sleep:  2 

schedule:  4 

boredom:  1 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight : 0 

' edu_train_e xpert : 
intro_accuracy : 4 

prob__of_success  : 
articulation:  3 


0 


kr_hp : 2 

computer:  2 

vehicles:  0 

weapon_systems : 
instruments:  4 

notation:  2 

test_equip:  3 

advises:  1 

answers : 1 

communicates:  1 

directs:  2 

indicates:  1 

informs : 2 

instructs:  4 

requests:  1 

transmits:  3 

supervises:  1 

negotiates:  3 

express_movement : 1 

interp_movement : 3 

namel : leadership 

name2 : medical_diag 

eq_count:  50  ~ 

sim  count:  86 


statistical:  0 

spatial:  3 

temporal : 2 

analogical:  1 

case_based:  0 

modeljbased:  2 

mathematical:  0 

deductive : 4 

inductive:  0 

means_ends : 1 

sub_goals : 0 

gen_and_test : 3 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : > 1 
acq_and_pres : 1 

form_proc_alg:  1 

decompose:  0 

recombine:  0 

generalize:  1 

specialize:  2 

ret_to_def : 2 

visual:  4 

verbal : 0 

auditory:  2 

kinesthetic:  1 

written:  1 

instrumentation:  4 

mm_interface : 0 

databases:  2 

historical:  1 

propadeutics : 1 

branching:  3 

dynamism:  0 

constraints:  1 

uncertainty:  2 

high_low_tech : 1 

quant_qual : 1 


complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  1 

gross:  1 

repetitive:  0 

'categorize:  2 

calculate:  2 

code:  0 

computerize:  1 

interpolate:  0 

itemize:  1 

learn:  4 

tabulate:  0 

translate:  3 

analyze:  0 

deduce:  4 

induce:  1 

choose:  1 

compare : 1 

compute : 1 

estimate:  1 

integrate:  2 

plan:  2 

supervise : 4 

monitor:  2 

interpret:  4 

facts:  1 

principles:  1 

procedures:  0 

analogs:  1 

cases_examples : 1 

oercep_speed:  0 

search_rec_inf o : 0 

id_ob j_act_events : 0 

scan_display : 0 

acceleration:  0 

confinement : 0 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  0 

preciseness : 0 

cog_attent : 0 

response_chaining : 0 

attention_span : 0 

sleep:  1 

schedule:  2 

boredom:  1 

general_health : 0 

age : 0 

gender : 0 

height:  0 

'eight:  0 

_^'du_train_expert : 2 

"intro_accuracy : 1 

prob_of_success : 1 

articulation:  1 

kr_hp : 0 


0 


computer : 2 

vehicles:  0 

we apon_sy stems : 
instruments : 2 

notation:  2 

test_equip:  2 

advises:  0 

answers : 2 

communicates:  0 

directs:  2 

indicates:  0 

informs : 0 

instructs:  3 

requests:  2 

transmits:  1 

supervises:  2 

negotiates:  3 

express_movement : 1 

interp_movement : 0 

namel:  leadership 

name2 : accounting 

eq_count:  64 

sim  count:  88 


statistical:  0 

spatial:  0 

temporal:  1 

analogical:  1 

case_based:  1 

model_based:  0 

mathematical:  4 

deductive : 0 

inductive : 2 

means_ends : 0 

sub_goals : 1 

gen_and_test : 3 

goal_vs_data : 1 

cover_and_dif f : 0 

prop_and_ref : 3 

acq_and_jpres : 1 

form_proc_alg : 1 

decompose:  1 

recombine : 2 

generalize:  1 

specialize:  3 

ret_to_def : 0 

visual:  0 

verbal:  1 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  0 

mm_interface:  4 

databases:  2 

historical:  1 

propadeutics : 0 

branching:  1 

dynamism:  1 

constraints:  1 

uncertainty:  0 

high_low_tech : 0 

quant_qual : 1 

complex__cont : 0 


compound:  0 

reflex:  0 

simple_disc : 0 

fine:  0 

gross:  0 

repetitive : 0 

^categorize : 0 

calculate : 2 

code:  0 

computerize:  3 

interpolate:  0 

itemize:  1 

learn:  3 

tabulate:  4 

translate : 1 

analyze:  4 

deduce : 0 

induce:  1 

choose:  2 

compare : 0 

compute : 3 

estimate:  1 

integrate : 1 

plan:  3 

supervise:  4 

monitor:  0 

interpret : 2 

facts:  2 

principles : 1 

procedures : 0 

analogs:  1 

cases_examples : 0 

percep_speed:  0 

search_rec_info : 0 

^ id_ob  j_act_e  ven  t s : 0 

scan_display : 0 

acceleration:  0 

confinement : 0 

isolation:  0 

contaminants:  0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain : 0 

preciseness:  4 

cog_attent:  0 

response_chaining : 0 

attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 2 

^ int ro_accuracy : 4 

prob_of_success : 3 

articulation:  4 

kr_hp:  0 

computer:  2 


0 


vehicles:  0 

weapon_systems : 
instruments:  0 

notation:  1 

test_equip:  0 

advises:  4 

answers : 0 

communicates:  0 

directs : 2 

indicates : 2 

informs:  2 

instructs:  4 

requests:  2 

transmits:  0 

supervises:  3 

negotiates:  3 

express_movement : 3 

interp_movement : 3 

namel:  leadership 

name2 : protocol_des 

eq_count : 64 

sim  count:  94 


statistical:  2 

spatial:  1 

temporal:  2 

analogical:  1 

case_based:  1 

model_based:  2 

mathematical:  2 

deductive : 0 

inductive : 1 

means_ends : 0 

sub_goals : 2 

gen_and_test : 1 

goal_vs_data : 0 

cover_and_dif f : 0 

prop_and_ref : 1 

acq_and_pres:  1 

form_proc_alg:  2 

decompose:  1 

recombine : 3 

generalize:  2 

specialize:  0 

ret_to_def:  1 

visual:  0 

verbal : 0 

auditory:  0 

kinesthetic:  0 

written:  0 

instrumentation:  2 

mm_interf ace : 0 

databases:  2 

historical:  1 

propadeutics : 2 

branching:  0 

dynamism:  0 

constraints:  0 

uncertainty:  1 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 0 

compound:  0 


reflex:  0 

simple_di.sc : 0 

fine:  0 

gross : 0 

repetitive:  0 

categorize:  2 

calculate : 1 

code : 0 

computerize:  1 

interpolate:  1 

itemize:  1 

learn:  2 

tabulate:  0 

translate:  4 

analyze:  4 

deduce : 0 

induce:  1 

choose:  1 

compare : 0 

compute : 2 

estimate:  1 

integrate:  1 

plan:  1 

supervise : 3 

monitor:  0 

interpret:  4 

facts:  0 

principles:  0 

procedures:  0 

analogs : 1 

cases_examples : 2 

percep_speed:  0 

search_rec_inf o : 
id_ob j_act_events : 
scan_display : 0 

acceleration:  0 

confinement : 0 

isolation:  0 

contaminants:  0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness:  0 

cog_attent : 0 

response_chaining : 
attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health : 0 

age : 0 

gender : 0 

height:  0 

weight : 0 

edu_train_expert : 
intro_accuracy : 4 

prob_of_success : 
articulation:  1 

kr_hp : 0 

computer : 0 

vehicles:  0 


0 


weapon_systems : 
instruments:  4 

notation:  2 

test_equip:  1 

advises : 4 

answers : 1 

communicates:  0 

directs:  1 

indicates:  2 

informs : 1 

instructs:  4 

requests:  1 

transmits:  0 

supervises:  0 

negotiates:  3 

express_movement : 3 

interp_movement : 3 

namel : surgery 

name2 : medical_diag 

eq_count : 57 

sim  count:  87 


statistical:  0 

spatial:  1 

temporal:  0 

analogical:  1 

case_based:  1 

model_based:  1 

mathematical : 0 

deductive:  1 

inductive : 0 

means=ends : 3 

sub_goals : 2 

gen_and_test : 0 

goal_vs_data : 1 

cover_and_dif f : 1 

prop_and_ref : 4 

acq_and__pres : 1 

form_proc_alg : 2 

decompose:  1 

recombine:  0 

generalize:  1 

specialize:  1 

ret_to_def : 1 

visual:  0 

verbal : 0 

auditory:  0 

kinesthetic:  3 

written:  0 

instrumentation:  0 

mm_interf ace : 1 

databases:  0 

historical:  0 

propadeutics:  0 

branching:  0 

dynamism:  2 

constraints:  1 

uncertainty:  1 

nigh_low_tech : 0 

quant_qual : 1 

complex_cont : 4 

compound:  2 

reflex:  0 


simple_disc:  4 

fine:  3 

gross:  3 

repetitive:  4 

categorize:  0 

calculate:  0 

code : 0 

computerize:  0 

interpolate:  0 

itemize:  2 

learn:  0 

tabulate:  0 

translate:  3 

analyze:  0 

deduce : 0 

induce : 0 

choose:  0 

compare : 1 

compute : 0 

estimate:  1 

integrate:  0 

plan:  3 

supervise:  4 

monitor:  2 

interpret : 0 

facts:  0 

principles:  0 

procedures:  0 

analogs : 0 

cases_examples : 1 

percep_speed:  2 

search_rec_inf o : 0 

id_obj_act_e vents : 4 

scan_display : 0 

acceleration:  0 

confinement : 2 

isolation:  0 

contaminants:  1 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  0 

fatigue:  2 

mental_strain : 1 

stress:  1 

phys_strain:  3 

preciseness:  4 

cog_attent : 4 

response_chaining:  2 

attention_span : 4 

sleep:  1 

schedule:  2 

boredom:  0 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 3 

prob_of_success : 0 

articulation : 2 

kr_hp : 2 

computer : 0 

vehicles:  0 

weapon_sy stems : 0 


instruments:  2 

notation:  0 

test_equip:  1 

advises : 1 

answers : 1 

communicates:  1 

directs:  0 

indicates:  1 

informs : 2 

instructs:  1 

requests:  1 

transmits:  2 

supervises:  3 

negotiates:  0 

express_movement : 0 

interp__movement : 3 

namel : surgery 

name2 : accounting 

eq_count:  36 

sim  count : 59 


statistical:  0 

spatial:  4 

temporal:  1 

analogical:  1 

case_based:  0 

model_based:  3 

mathematical:  4 

deductive : 3 

inductive : 2 

means_ends : 4 

sub_goals : 1 

gen_and_test : 0 

goal_vs_data:  1 

cover_and_dif f : 2 

prop_and_ref : 0 

acq_and_pres : 1 

f orm_proc_alg : 0 

decompose : 2 

recombine:  2 

generalize:  1 

specialize:  0 

ret_to_def:  1 

visual:  4 

verbal:  1 

auditory:  2 

kinesthetic:  4 

written:  0 

instrumentation:  4 

mm^interf ace : 3 

databases : 4 

historical:  0 

propadeutics : 1 

branching:  4 

dynamism:  1 

constraints:  1 

uncertainty:  1 

high_low_tech : 1 

quant_qual : 1 

complex_cont : 4 

compound:  2 

reflex:  0 

simple_disc:  4 


fine:  4 

gross:  4 

repetitive : 4 

categorize:  2 

calculate : 4 

code : 0 

.computerize:  4 

interpolate : 0 

itemize:  4 

learn : 1 

tabulate:  4 

translate : 1 

analyze:  4 

deduce : 4 

induce:  2 

choose:  3 

compare:  0 

compute : 4 

estimate : 3 

integrate:  3 

plan:  4 

supervise : 4 

monitor:  4 

interpret : 2 

facts:  3 

principles:  2 

procedures:  0 

analogs : 0 

cases_examples : 0 

percep_speed:  2 

search_rec_inf o : 0 

id_ob  j_act_events  : 4 

scan_display : 0 

acceleration:  0 

„ confinement : 2 

"isolation : 0 

contaminants : 1 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  0 

fatigue:  3 

mental_strain : 3 

stress:  4 

phys_strain:  3 

preciseness:  0 

cog_attent:  4 

response_chaining : 2 

attention_span:  4 

sleep:  2 

schedule:  4 

boredom:  1 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 4 

articulation : 1 

x^,;r_hp:  2 

^"computer : 4 

vehicles:  0 

weapon_systems : 0 

instruments:  4 


notation:  1 

test_equip:  3 

advises : 3 

answers:  1 

communicates:  1 

directs:  4 

indicates:  3 

informs : 0 

instructs:  0 

requests:  1 

transmits:  3 

supervises:  4 

negotiates:  0 

express_movement : 2 

interp_movement : 0 

namel : surgery 

name2 : protocol_des 

eq_count:  44 

sim  count:  61 


statistical:  2 

spatial:  3 

temporal:  0 

analogical:  1 

case_based:  2 

model_based:  1 

mathematical : 2 

deductive:  3 

inductive:  1 

means_ends : 4 

sub_goals : 0 

gen_and_test : 4 

goal_vs_data : 0 

cover_and_dif f : 2 

prop_and_ref : 4 

acq_and_pres : 1 

form_proc_alg:  3 

decompose:  0 

recombine:  3 

generalize:  2 

specialize:  3 

ret_to_def : 0 

visual:  4 

verbal:  0 

auditory:  2 

kinesthetic:  4 

written:  1 

instrumentation:  2 

mm_interf ace : 1 

databases:  0 

historical:  0 

propadeutics : 3 

branching:  3 

dynamism:  2 

constraints : 2 

uncertainty:  2 

high_low_tech : 0 

quant_qual : 1 

complex_cont : 4 

compound:  2 

reflex:  0 

simple_disc:  4 

fine:  4 


gross:  4 

repetitive:  4 

categorize:  0 

calculate:  3 

code : 0 

computerize:  0 

^^interpolate : 1 

itemize:  4 

learn:  2 

tabulate:  0 

translate:  4 

analyze:  4 

deduce : 4 

induce:  0 

choose:  0 

compare : 0 

compute : 3 

estimate:  1 

integrate:  3 

plan:  0 

supervise:  3 

monitor:  4 

interpret : 0 

facts:  1 

principles:  1 

procedures:  0 

analogs : 0 

cases_examples : 2 

percep_speed:  2 

search_rec_inf o : 0 

id_ob  j_act_events : 4 

scan_display : 0 

acceleration:  0 

confinement : 2 

N ^isolation:  0 

contaminants:  1 

electricity:  0 

lighting:  2 

magnetism:  0 

noise:  0 

fatigue:  3 

mental_strain : 3 

stress:  4 

phys_strain:  3 

preciseness : 4 

cog_attent : 4 

response_chaining : 2 

attention_span : 4 

sleep:  2 

schedule:  4 

boredom : 1 

general_health : 0 

age:  0 

gender : 0 

height:  0 

weight : 0 

edu_train_expert : 0 

intro_accuracy : 0 

prob_of_success : 0 

articulation:  2 

\r_hp:  2 

v computer : 2 

vehicles:  0 

weapon_systems : 0 

instruments : 0 

notation:  4 


test_equip:  2 

advises:  3 

answers : 0 

communicates:  1 

directs:  1 

indicates:  3 

informs : 1 

instructs:  0 

requests:  0 

transmits:  3 

supervises:  1 

negotiates:  0 

express_movement : 2 

interp_movement : 0 

namel:  medical_diag 

name2 : accounting 

eq_count:  46 

sim  count:  71 


statistical:  0 

spatial:  3 

temporal:  1 

analogical:  0 

case_based:  1 

model_based:  2 

mathematical:  4 

deductive : 4 

inductive:  2 

means_ends : 1 

sub_goals : 1 

gen_and_test : 0 

goal_vs_data : 0 

cover_and_dif f : 1 

prop_and_ref : 4 

acq_and_pres : 0 

f orm_proc_alg : 2 

decompose:  1 

recombine:  2 

generalize:  2 

specialize:  1 

ret_to_def:  2 

visual:  4 

verbal:  1 

auditory:  2 

kinesthetic:  1 

written:  0 

instrumentation:  4 

mm_interf ace : 4 

databases:  4 

historical:  0 

propadeutics:  1 

branching:  4 

dynamism:  1 

constraints:  0 

uncertainty:  2 

high_low_tech : 1 

quant_qual : 2 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  1 

gross:  1 


repetitive : 0 

categorize:  2 

calculate : 4 

code : 0 

computerize:  4 

interpolate : 0 

itemize : 2 

learn : 1 

tabulate:  4 

translate : 2 

analyze:  4 

deduce:  4 

induce:  2 

choose:  3 

compare : 1 

compute:  4 

estimate:  2 

integrate : 3 

plan:  1 

supervise : 0 

monitor:  2 

interpret:  2 

facts:  3 

principles : 2 

procedures:  0 

analogs : 0 

cases_examples : 1 

percep__ speed:  0 

search_rec_inf o : 
id_obj_act__e  vents : 
scan_display : 0 

acceleration : 0 

confinement:  0 

isolation : 0 

contaminant  s : 0 

electricity:  0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain:  0 

preciseness:  4 

cog_attent : 0 

response_chaining : 
attention_span : 0 

sleep:  1 

schedule:  2 

boredom:  1 

general_health : 0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 
intro_accuracy : 3 

prob_of_success : 
articulation : 3 

kr__hp : 0 

computer:  4 

vehicles:  0 

weapon_sy stems : 0 

instruments:  2 

notation:  1 

test_equip:  2 


advises:  4 

answers : 2 

communicates:  0 

directs:  4 

indicates:  2 

informs : 2 

instructs:  1 

requests:  0 

transmits:  1 

supervises:  1 

negotiates:  0 

express_movement : 2 

interp_movement : 3 

namel : medical_diag 

name2 : protocol_des 

eq_count:  57 

sim  count:  88 


statistical:  2 

spatial:  2 

temporal:  0 

analogical:  0 

case_based:  1 

model_based:  0 

mathematical:  2 

deductive : 4 

inductive:  1 

means_ends : 1 

sub_goals : 2 

gen_and_test : 4 

goal_vs_data:  1 

cover_and_dif f : 1 

prop_and_ref : 0 

acq_and_pres : 0 

form_proc_alg:  1 

decompose : 1 

recombine : 3 

generalize:  1 

specialize:  2 

ret_to_def : 1 

visual:  4 

verbal:  0 

auditory:  2 

kinesthetic:  1 

written:  1 

instrumentation:  2 

mm_interface:  0 

databases : 0 

historical:  0 

propadeutics : 3 

branching:  3 

dynamism:  0 

constraints:  1 

uncertainty:  3 

high_low_tech : 0 

quant_qual : 0 

complex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  1 

gross:  1 

repetitive:  0. 


categorize:  0 

calculate : 3 

code : 0 

computerize : 0 

interpolate : 1 

itemize:  2 

learn : 2 

tabulate:  0 

translate : 1 

analyze:  4 

deduce:  4 

induce:  0 

choose:  0 

compare : 1 

compute:  3 

estimate:  0 

integrate : 3 

plan:  3 

supervise : 1 

monitor:  2 

interpret : 0 

facts:  1 

principles:  1 

procedures : 0 

analogs:  0 

cases_examples : 1 

percep_speed:  0 

search_rec_info : 0 

id_ob j_act_e vents : 0 

scan_display : 0 

acceleration:  0 . 

confinement:  0 

isolation : 0 

contaminants:  0 

^^^electricity : 0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  1 

mental_strain : 2 

stress:  3 

phys_strain : 0 

preciseness:  0 

cog_attent : 0 

response_chaining : 0 

attention_span : 0 

sleep:  1 

schedule:  2 

boredom:  1 

general_health : 0 

age : 0 

gender : 0 

height : 0 

weight : 0 

edu_train_expert : 0 

1 intro_accuracy : 3 

prob_of__success : 0 

articulation:  0 

kr__hp : 0 

computer:  2 

: vehicles : 0 

rfeapon_systems : 0 

instruments : 2 

notation:  4 

test_ equip:  1 

advises:  4 


answers : 1 

communicates:  0 

directs:  1 

indicates:  2 

informs : 1 

instructs:  1 

requests:  1 

transmits:  1 

supervises:  2 

negotiates:  0 

express_movement : 2 

interp_movement : 3 

namel:  accounting 

name2 : protocol_des 

eq_count:  65 

sim  count:  88 


statistical:  2 

spatial : 1 

temporal:  1 

analogical:  0 

casejbased:  2 

model_based:  2 

mathematical:  2 

deductive:  0 

inductive : 3 

means_ends : 0 

sub_goals : 1 

gen_and_test : 4 

goal_vs_data:  1 

cover_and_diff : 0 

prop_and_ref : 4 

acq_and__pres : 0 

form_proc_alg:  3 

decompose:  2 

recombine:  1 

generalize:  3 

specialize:  3 

ret_to_def:  1 

visual:  0 

verbal:  1 

auditory:  0 

kinesthetic:  0 

written:  1 

instrumentation:  2 

mm_interface :.  4 

databases:  4 

historical:  0 

propadeutics : 2 

branching:  1 

dynamism:  1 

constraints:  1 

uncertainty:  1 

high_low_tech : 1 

quant_qual : 2 

compl'ex_cont : 0 

compound:  0 

reflex:  0 

simple_disc:  0 

fine:  0 

gross : 0 

repetitive:  0 

categorize:  2 


calculate : 1 

code:  0 

computerize:  4 

interpolate:  1 

itemize:  0 

learn : 1 

tabulate:  4 

translate : 3 

analyze:  0 

deduce : 0 

induce:  2 

choose:  3 

compare : 0 

compute:  1 

estimate:  2 

integrate : 0 

plan:  4 

supervise : 1 

monitor:  0 

interpret : 2 

facts:  2 

principles:  1 

procedures:  0 

analogs : 0 

cases_examples : 2 

percep_speed:  0 

search_rec_inf o : 
id_obj_act_e  vents : 
scan_display : 0 

acceleration:  0 

confinement:  0 

isolation : 0 

contaminant  s : 0 

electricity : 0 

lighting:  0 

magnetism:  0 

noise:  0 

fatigue:  0 

mental_strain : 0 

stress:  0 

phys_strain:  0 

preciseness:  4 

cog_ attent : 0 

response_chaining : 
attention_span : 0 

sleep:  0 

schedule:  0 

boredom:  0 

general_health:  0 

age : 0 

gender:  0 

height : 0 

weight : 0 

edu_train_expert : 
intro_accuracy : 0 

prob__of_success : 
articulation:  3 

kr_hp:  0 

computer:  2 

vehicles:  0 

weapon_systems : 0 

instruments:  4 

notation:  3 

test_equip:  1 

advises:  0 

answers : 1 


communicates:  0 

directs:  3 

indicates:  0 

informs : 1 

instructs:  0 

requests:  1 

transmits:  0 

supervises:  3 

negotiates:  0 

express_movement : 0 

interp_movement : 0 
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1 


SAS 


14:03  FRIDAY,  MARCH  29, 


INITIAL  FACTOR  METHOD:  PRINCIPAL  COMPONENTS 


PRIOR  COMMONALITY  ESTIMATES:  ONE 

EIGENVALUES  OF  THE  CORRELATION  MATRIX:  TOTAL  = 36  AVERAGE  = 1 


1 

2 

3 

4 

5 

6 

7 

a 

9 

10 

11 

EIGENVALUE 

9 , 3 o 11  7 4 

5 . 955761 

J . 8 30a  77 

3.075796 

i . 7 b b a a a 

2 . 1 319  74 

1.863212 

l . 52  3088 

1 

.233815 

1.068160 

0.847205 

DIFFERENCE 

3.405712 

2.125084 

0 . 754881 

0.308908 

0.531914 

0.371762 

0.340123 

0 .289273 

0 

.165655 

0.220955 

0.115349 

PROPORTION 

0 . 2600 

0 .1654 

0.1064 

0.0854 

0.0769 

0 .0621 

0.0518 

0.0423 

0.0343 

0.0297 

0.0235 

CUMULATIVE 

0 . 2600 

0.4255 

0.5319 

0.6173 

0.6942 

0.7563 

0.8080 

0.8503 

0.8846 

0.9143 

0.9378 

13 

14 

15 

16 

17 

18 

19 

20 

21 

2 2 

23 

EIGENVALUE 

0.642877 

0.563419 

0.300796 

0 . 000000 

0.000000 

0.000000 

0 . 000000 

o . oooooo 

0 , 

.000000 

o . oooooo 

0.000000 

DIFFERENCE 

0.079458 

0.262621 

0 .300798 

0 .000000 

0.000000 

0.000000 

0 . 000000 

0 .oooooo 

0 

. oooooo 

o . oooooo 

0 .oooooo 

PROPORTION 

0.0179 

0 , 015  7 

0.0084 

0 . 0000 

0 . 0000 

0.0000 

0.0000 

0 .0000 

0 .0000 

0 . 0000 

0.0000 

CUMULATIVE 

0.9760 

0.9916 

1 .0000 

1 .0000 

1 . 0000 

1 . 0000 

1 . 0000 

1 . 0000 

1.0000 

1 . 0000 

1.0000 

25 

2 b 

27 

28 

29 

30 

3 1 

32 

3 3 

3 4 

35 

EIGENVALUE 

0 . 000000 

0.0000  00 

0.000000 

0 . 000000 

0 . 000000 

0 . 000000 

0 . oooooo 

0.000000 

0 . 

. oooooo 

0 .oooooo 

0 .oooooo 

DIFFERENCE 

0 . 000000 

0 . 000000 

0 .000000 

0.000000 

0.000000 

0.000000 

0 . oooooo 

o . oooooo 

0 

.oooooo 

0 .oooooo 

0 .oooooo 

PROPORTION 

0 . 0000 

0.0000 

0 .0000 

0 .0000 

0 .0000 

0 . 0000 

0.0000 

0 . 0000 

0 .0000 

0 .0000 

0.0000 

CUMULATIVE 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1 .0000 

1.0000 

1 .00  00 

3 FACTORS  WILL  BE  RETAINED  BY  THE  NFACTOH  CRITERION 

1 SAS  14:03  FRIDAY,  MARCH  29, 


INITIAL  FACTOR  METHOD:  PRINCIPAL  COMPONENTS 


FACTOR  PATTERN 


FACTOR 1 


FACTOR  2 


FACTOR  3 


« 


VI 

-0,40103 

0 .11303 

0.66897 

V 2 

0 .61216 

-0.13073 

0 .37429 

V 3 

0 .41547 

-0.11237 

0.35918 

V 4 

0 . 1 9969 

0.48374 

-0 .08395 

V 5 

-0.04236 

0.59670 

-0.43581 

V 6 

0 . 4 46  17 

0.0  7582 

0.65240 

V 7 

-0.21463 

-0 . 50372 

0 . 23608 

V 8 

0 .71555 

0 . 10747 

0 . 01362 

V9 

-0 .33142 

0 .43877 

0.67015 

VlO 

0 . 50232 

0 .20790 

-0 . 06619 

Vll 

-0 .00730 

0 . 26182 

-0.13447 

V 1 2 

-0 .52665 

0 .49165 

-0.08424 

V 1 3 

0.11278 

-0 .63458 

0 .04  274 

VI  4 

0 . 31709 

0 .22569 

-0.12649 

V 1 5 

-0.45331 

0 .68533 

-0.15275 

VI  6 

-0 . 39003 

0.41726 

0.43918 

V17 

0 .16207 

-0.59879 

-0 . 09150 

VI  8 

0.22030 

0.66261 

-0.44309 

V 1 9 

-0.67393 

0.21869 

0 . 36803 

V 2 0 

-0 . 35066 

0.54202 

0.58100 

V21 

0 .66178 

-0 .06360 

-0 .00712 

V 2 2 

-0 .07640 

0.03127 

-0 . 36567 

INPUTS 

0.71659 

0 .10465 

-0,01318 

c 


1991  1 


12 

0.731856 
0.088979 
0.0203 
0 .9581 

2 4 

0 . 000000 
0 . 000000 
0.0000 
1.0000 

36 

0 . 000000 

0.0000 
1 .0000 


1991  2 


( 
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( 
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COMPLEX 

TECH 

INFORM 

SOLVE 

RECALL 

PERCEPT 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAIN 


0 .71250 
-0.23455 
-0 .24569 
0 . 57623 
0 .17473 
0.81591 
0.75149 
0 .92888 
0 .74617 
0 .41503 
0.67036 
0.35911 
0.86518 


0.13723 
-0.45734 
-0 .28452 
0 . 40924 
0 . 76963 
-0.12420 
-0  .34170 
-0 .03177 
-0 .06245 
0 . 72614 
-0.04855 
0 . 68584 
0.21844 


0 . 21324 
0 ,41959 
0 .37996 
0 . 46531 
0.22225 
0 .19984 
-0.18207 
-0.04170 
0 .05883 
0.11054 
0.23426 
-0.32230 
0.17353 


VARIANCE  EXPLAINED  BY  EACH  FACTOR 


(f 

Page  2 


FACTOR1  FACTOR  2 FACTOR! 
9.361474  5.955761  3.830677 


1 


FINAL  COMMUNALITY  ESTIMATES:  TOTAL  = 14.147912 


VI 

0.62112a 

V 2 

0.531923 

V 3 

0.314251 

V 4 

0.280925 

V 5 

0.547777 

V 6 

0.630445 

VI  3 
0.417235 

V 1 4 
0 . 167485 

V 1 5 
0.698503 

VI  6 
0.519111 

V 1 7 
0.393184 

VI  8 
0.683921 
SAS 

V 7 

0 . 355534 

V 8 

0 . 5 2 3 7 5 0 

V 9 

0.751453 

V10 
0 . 299929 

VI  1 
0.086684 

VI  2 
0.526181 

VI  9 
0 .637452 

V 2 0 
0 . 754317 

V 2 1 
0.442050 

V 2 2 
0.140527 
14:03 

INPUTS  COMPLEX 

0.524627  0 . 571963 

FRIDAY,  MARCH  29,  1991 

INITIAL  FACTOR  METHOD:  PRINCIPAL  COMPONENTS 

TECH 

0.440228 

INFORM 
0 . 285683 

SOLVE 
0 . 716030 

RECALL 
0 .672264 

PERCEPT 
0 . 721074 

ENVIRON  PSYCHE 

0.714646  0 .865568 

SAS 

PHYSICAL 
0 . 564124 

PSYCH  EQUIP  COMMUNIC  DOMAIN 

Q. 711749  0.506619  0.703203  0.826368 

14:03  FRIDAY,  MARCH  29,  1991  4 


ROTATION  METHOD:  VARIMAX 


ORTHOGONAL  TRANSFORMATION  MATRIX 

12  3 


1 

0.92649 

0 . 18406 

-O .32822 

2 

0 .00274 

0 . 86889 

0 . 49500 

3 

0.37630 

-0 . 45952 

0.80452 

ROTATED  FACTOR  PATTERN 

FACTOR1 

FACTOR2 

FACTOR J 

VI 

-0 . 11951 

-0 . 28301 

0 . 72578 

V 2 

0 .70765 

-0 . 17290 

0 .03549 

V 3 

0.51978 

-0 .18621 

0 .09697 

V 4 

0.15475 

0 .49565 

0.10637 

V5 

-0 .20160 

0 .71093 

-0 .04134 

V6 

0.65908 

-0.15179 

0 .41595 

V7 

-0.11140 

-0.58566 

0.01104 

V 8 

0.66838 

0 . 21883 

-0 . 17070 

V9 

-0.05368 

0.01229 

0 .86511 

VI  0 

0 . 44106 

0.30352 

-0.11521 
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VI  1 

- 0 H 0 5 6 o 4 

0.28794 

0.02381 

VI  2 

-0.51829 

0 . 36896 

0 .34846 

VI  3 

0 .118  ft 4 

-0.55026 

-0 . 31675 

VI  4 

0 .24681 

0.31259 

-0.09412 

VI  5 

-0.47559 

0.58223 

0 . 3 6 513 

V 1 6 

-0 . 19495 

0.08896 

0.68789 

Vl  7 

0.11408 

-0.44840 

-0.42321 

V 1 8 

0 .03919 

0.81989 

-0 . 10079 

V 1 9 

-0 .48530 

-0 . 10,314 

0 .62554 

V20 

-0 . 10477 

0 . 13943 

0 .85082 

V21 

0.61029 

0.06982 

-0 . 2544  1 

V22 

-0 . 20330 

0.18114 

-0  . 25363 

INPUTS 

0 .65925 

0.22888 

-0 .19400 

COMPLEX 

0 . 74075 

0.15240 

0 . 00563 

TECH 

-0 .06067 

-0.63336 

0.18817 

INFORM 

-0 . 08543 

-0 .46703 

0.24549 

SOLVE 

0 . 71009 

0.24783 

0 . 38779 

RECALL 

0. 24763 

0 . 59876 

050243 

PERCEPT 

0. 83030 

-0 . 04956 

-0  . 16851 

ENVIRON 

0 .62680 

-0 .07492 

-0 . 56227 

PSYCHE 

0.84483 

0.16253 

-0 . 35415 

PHYSICAL 

0.71328 

0 . 05604 

-0  . 22849 

PSYCH 

0.42811 

0.65653 

0.31215 

EQUIP 

0 . 7 0 9 1 0 

-0.02644 

-0.05559 

COMMUNI C 

0.21331 

0 . 81011 

-0.03767 

DOMAIN 

0.86749 

0 . 26930 

-0.03624 

VARIANCE 

EXPLAINED 

BY  EACH 

FACTOR 

FACTOR1  FACTOR  2 FACTOR  3 
8.578238  5.622419  4.947205 

1 SAS  14:03  FRIDAY,  MARCH  29,  1991  5 

ROTATION  METHOD:  VAR I MAX 


FINAL  COMMONALITY  ESTIMATES:  TOTAL  - 19.147912 


VI 

0.621128 

0 . 

V 2 

.531923 

0 

V 3 

.314251 

V 4 

0 . 280925 

V 5 

0 . 547777 

V6 

0 .630445 

V7 

U . 355534 

0 . 

V 8 

, 52  3 750 

0 . 

V9 

. 751453 

0 

V10 

.299929 

0 

Vll 

.086684 

V 1 2 
0 . 526181 

V 1 3 
0.417235 

0 , 

VI  4 
. 167485 

0 . 

VIS 

.698503 

V 1 6 
0.519111 

VI  7 
0.393184 

vi  e 
0683921 

V 1 9 
0 . u37452 

0 . 

V20 
, 754317 

0 . 

V21 
, 442050 

0 

V22 
. 140527 

0 . 

INPUTS 
. 524627 

COMPLEX 
0 - 571963 

TECH 

0.440228 

0 , 

INFORM 

.285683 

0 . 

SOLVE 

.716030 

RECALL 
0 .672264 

PERCEPT 
0 .721074 

ENVIRON 

0.714646 

PSYCHE 
0 . 865568 

PHYSICAL 
0. 564124 

0 . 

PSYCH 

.711749 

0 

EQUIP 
. 506619 

COMMUNIC 
0 . 703203 

DOMAIN 
0 .826368 

<s  # < 
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AX  VAX  $ DUA1  : [ AX  . KATAX  ] CLUSTER  i . US  ; 1 
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S AS 


9:39  TUESDAY , APRIL  2 


CENTROID  HIERARCHICAL  CLUSTER  ANALYSIS 


EIGENVALUES  OF  THE  COVARIANCE  MATRIX 


EIGENVALUE 

DIFFERENCE 

PROPORTION 

CUMULATIVE 

I 

1 3 . 894  L 

6.26352 

0 . 2 8 a 7 9 7 

0.28880 

2 

7 .6306 

2 .45736 

0.158606 

0 . 44740 

3 

5.1732 

0.57925 

0 .107529 

0 . 55493 

4 

4 . 5940 

1 .07205 

0.095489 

0 .65042 

5 

3.5219 

0.4279  1 

0.073205 

0.72363 

6 

3.0940 

0.57024 

0.064311 

0.78794 

7 

2.52  38 

0.77827 

0.052458 

0.84039 

8 

1.7455 

0.49820 

0.036281 

0 .87663 

9 

1.2473 

0.03642 

0.025926 

0 .90260 

10 

1 .2109 

0.1  2309 

0.025169 

0 .92777 

11 

1 . 0878 

0.14892 

0 . 0 2261  0 

0.95038 

12 

0.9389 

0 .30811 

0.019515 

0.96990 

13 

0 .6308 

0.12608 

0.013111 

0.98301 

14 

0 . 5047 

0.19184 

0.010490 

0.99350 

15 

0.3128 

0.31284 

0.006503 

1 .00000 

16 

0 . 0000 

0 . 00000 

0.000000 

1 . ooooo 

17 

0.0000 

0 .00000 

0.000000 

1 .00000 

18 

0 .0000 

0 .00000 

0 .000000 

1 .00000 

19 

0.0000 

0 .00000 

0 .000000 

1 .00000 

20 

0 .0000 

0 .00000 

0.000000 

1 .00000 

21 

0 .0000 

0 .00000 

0.000000 

1 . 00000 

22 

0.0000 

0 .00000 

0.000000 

1.00000 

23 

0.0000 

0 . 00000 

0.000000 

1 .00000 

24 

0.0000 

0.00000 

0 .000000 

1 .00000 

25 

0 . 0000 

0 .00000 

0 .000000 

1 .00000 

26 

0.0000 

0 . 00000 

0 .000000 

1 .00000 

27 

0 . 0000 

0 .00000 

0 .000000 

1 .00000 

28 

-0 .0000 

0 . ooooo 

- . 000000 

1 .00000 

29 

-0  . 0000 

0 . ooooo 

-.000000 

1 .00000 

30 

-0.0000 

0 . 0 0"  DO 

- . 000000 

1.00000 

31 

-0  . 0 0 0 Q 

o . ooooo 

- . 000000 

1 .00000 

32 

-0 .0000 

0 . ooooo 

- .000000 

1 .00000 

33 

-0  . 0000 

0 . ooooo 

- . 000000 

1 .00000 

34 

-0 . 000G 

0 . ooooo 

- . 000000 

1 .00000 

35 

-0 .0000 

0 .00000 

- .000000 

1 .00000 

36 

-0 . 0000 

- . 000000 

1.00000 

ROOT-MEAN -SQUARE 

TOTAL— SAMPLE  STANDARD  DEVIATION 

- 1 . 1 560  J 

ROOT-MEAN -SQUARE 

DISTANCE  BETWEEN 

OBSERVATIONS 

= 9.8092 

NUMBER 

FREQUENCY 

NORMALIZED 

OF 

OF  NEW 

CENTROID 

CLUSTERS  CLUSTERS 

JOINED 

CLUSTER 

DISTANCE 

15  sw  da  s l g 

proton  ul 

2 

0 , 578  896 

14  pilot 

t r 

surgery 

2 

0 .62223  3 

13  cargo 

lo 

account i 

2 

0.666616 

12  program 

leader sh 

2 

0.671527 

1 1 equ ip  di 

CL12 

3 

0 .663370 

10 


CL1  1 


0 .662742 


, 1991 


( 


( 


A I VAX $ DU A 1 : ( A I . KATAX 1 CLUSTER 3 . LIS ; 1 
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1 


1 


e 

q 

u 

i 

p 

d 

i 

|j  . 9 4- 


9 

CLIO 

d r a i r^ge 

6 

0.673242 

8 

CL9 

f o ra_f i 1 

7 

0.638162 

SAS 


CENTROID 

HIERARCHICAL 

CLUSTER  ANALYSIS 

NUMBER 

FREQUENCY 

NORMALIZED 

or 

OF  NEW 

CENTROID 

CLUSTERS 

CLUSTERS 

JOINED 

CLUSTER 

DISTANCE 

7 

CL8 

CL1  5 

9 

0 . 

. 722  4 3 3 

6 

CL7 

nedica 1 

1 0 

0 

.732175 

5 

CL6 

language 

11 

0 

.783836 

4 

CL1  4 

air  t raf 

3 

0 

. 799688 

3 

CL5 

wea the r 

12 

0 

.825345 

2 

CL3 

CL4 

15 

0 

.822499 

1 

CL2 

console 

16 

0 

.811065 

SAS 


CENTROID 

HIERARCHICAL 

CLUSTER 

ANALYSIS 

NAME 

OF  OBSERVATION 

OR  CLUSTER 

P 

1 

c 

a 

d 

t 

s 

P 

in 

1 

r 

a 

a 

c 

r 

o 

w 

r 

e 

a 

w 

o 

a 

r 

c 

a 

r 

o 

d 

n 

a 

9 

d 

9 

o 

a 

d 

t 

i 

9 

a 

r 

a 

o 

u 

r 

a 

o 

c 

u 

t 

a 

r 

n 

f 

s 

c 

a 

a 

h 

a 

s 

T 

t 

9 

i 

i 

o 

1 

9 

a 

h 

o 

i 

a 

1 

9 

1 

a 

r 

( 


9:39  TUESDAY,  APRIL 


9:40  TUESDAY,  APRIL 


P 

i 

I 

o 

t 

t 

r 


s 

u 

r 

9 

a 

r 

Y 


a 

1 

r 

t 

r 

a 

t 


c 

o 

n 

s 

0 

1 
a 


D 

I 

S 

T 

A 

N 

C 

E 

B 

E 

T 

W 

E 

E 

N 


0 . a + XXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXX XXXXXXXXXXX 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 


0 . 7 


0 - h 


0.5  ♦ 


xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxx xxxxxx xxxxxxx 


+XXXXXXX XXXXXXX XXX XXXXXXXXXXX XXXXXX XXX 
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX 
XXXXXX.  .XXXXXXXXXXXX  . XXXXXX 

xxxxxx 


xxxxxxx 

xxxxxxx 

xxxxxxx 

xxxxxxx 

xxxxxxx 

xxxxxxx 

xxxxxxx 


xxxxxxxxxxxxx 
xxxxxxx 
xxxxxxx 
xxxxxxx 
xxxxxxx 
xxxxxxx 
xxxxxxx 
xxxxxxx 
xxxxxxx 
xxxxxxx 


C I 

L I 

U 0.44“ 

S I 
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2,  1991  2 


2 , 1991  3 
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T J 

E | 

R | 

0.3  + 

C I 

E j 

N I 

T [ 

R 0.2* 

0 | 

1 I 

D j 

S j 

0.1  f 
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1 Average  scores  for  eight  clusters  9:40  TUESDAY,  APRIL  2,  1991  4 

CLUSTER^ 1 

TASK  VI  V 2 V 3 V4  V5  V6  V7  V8  V9 

swedes ig  004033102 

protocol  2 1 4 0 4 2 2 0 3 


TASK 

V10 

VI  1 

V 1 2 

VI  3 

V 1 4 

V 1 5 

Vl  6 

V 1 7 

via 

s w de s l g 

4 

4 

4 

1 

0 

4 

4 

2 

4 

protocol 

0 

4 

4 

1 

0 

4 

4 

1 

4 

TASK 

VI  9 

V 2 0 

V 2 1 

V 2 2 

INPUTS 

COMPLEX 

TECH 

INFORM 

SOLVE 

s w de  s l g 

4 

2 

0 

1 

0 .90000 

1 

.25000 

1.50000 

1.44444 

2.08333 

protocol 

3 

3 

0 

1 

1 . 70000 

0 

.25000 

2 . 00000 

2 .00000 

2.16667 

TASK 

RECALL 

PERCEPT 

ENVIRON 

PSYCHE 

PHYS  J-CAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAIN 

s w de  s i g 

2 .40000 

0 

0 

0 

0 

1 . 

.50000 

1 . 3 3 3 3 3 

0.69231 

1 

p ro  t oco 1 

2 .80000 

0 

0 

0 

0 

2 . 

. 50000 

1 . 166  67 

1.61538 

1 

— 

CLUSTER  — 2 

TASK 

VI 

V 2 

V 3 

V 4 

V 5 

V 6 

V 7 

v a 

V 9 

pilot  t r 

0 

4 

4 

3 

3 

3 

0 

4 

1 

surgery 

0 

4 

4 

1 

2 

3 

0 

3 

2 

TASK 

VI  0 

Vll 

V 1 2 

VI  3 

VI  4 

V l 5 

V 1 6 

V 1 7 

via 

pi lot_t r 

3 

4 

0 

3 

0 

0 

4 

4 

3 

surge  ry 

4 

4 

0 

1 

2 

0 

3 

4 

4 

TASK 

VI  9 

V 2 0 

V 2 1 

V 2 2 

INPUTS 

COMPLEX 

TECH 

INFORM 

SOLVE 

C ( c 


mini  I II 


( 

\ 

AI VAX $ DUAl  : [AI  . KAT AX ] C LUSTE R 3 .LIS; 1 

p i 1 o t_t  r 0 2 3 

surgery  0 * 3 

TASK  RECALL  PERCEPT  ENVIRON 

pilotjc  2.40000  3.50000  1.83  33  3 

surgery  2.00000  1.50000  0.83333 

Ave 


TASK 

VI 

V2 

V 3 

cargo  1 o 

0 

4 

4 

account i 

0 

0 

3 

p r og  r a«_ 

0 

0 

4 

leadersh 

0 

0 

2 

equip  di 

0 

1 

1 

drai r_ge 

1 

0 

2 

TASK 

V10 

VI 1 

VI  2 

cargo  1 o 

3 

4 

0 

acc  o un  t i 

0 

3 

0 

p rog  ran 

0 

4 

0 

leadersh 

0 

2 

3 

equip  di 

0 

3 

1 

drai r_ge 

0 

2 

0 

TASK 

V 1 9 

V 2 0 

V 2 1 

ca  rgo^i o 

1 

0 

2 

account! 

2 

0 

3 

progran_ 

0 

0 

0 

leade r sh 

0 

1 

0 

•quip_di 

1 

0 

0 

drai rge 

3 

3 

0 

TASK 

RECALL 

PERCEPT 

ENVIRON 

cargo  1 o 

2-40000 

0 

1.00000 

account i 

1 . 80000 

0 

0 . 00000 

prograi^ 

2 . 40000 

0 

0 . 00000 

leadersh 

2 .60000 

0 

0.00000 

equip_di 

1 .60000 

0 

1 . 50000 

drai r_ge 

1 . 8000Q 

0 

0 . ooooo 

TASK 

VI 

V2 

V 3 

fori^f i 1 

0 

0 

2 

TASK 

V10 

Vll 

V 1 2 

fora  £ i 1 

2 

4 

3 
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( 


1 

2 - 30000 

4 . 

.00000 

1 

2 .80000 

2 

. 50000 

PSYCHE 

PHYSICAL 

PSYCH 

2-18182 

2.60000 

3 

.25000 

3 . 0909  1 

0 . 00000 

2 

.00000 

scores 

for  eight  clusta 

t s 

2.00000  0.555556  2-41667 

1.50000  0.888889  3.00000 

EQUIP  COMMUNIC  DOMAIN 

1.66667  2.84615  3 

1.83333  2.00000  3 

9:40  TUESDAY , APRIL  2,  1991  5 


CLUSTER=  3 


- V 4 

V 5 

V 6 

0 

2 

1 

0 

2 

0 

0 

3 

0 

l 

3 

0 

0 

2 

1 

1 

1 

0 

V 1 3 

VI  4 

V 1 5 

3 

0 

0 

2 

0 

0 

1 

0 

3 

1 

0 

3 

3 

0 

0 

3 

0 

0 

V 2 2 

INPUTS 

COMPLEX 

2 

0 . 90000 

0 . 75000 

0 

2.10000 

0 . 75000 

3 

1.50000 

2 . 00000 

0 

1.40000 

0 . 50000 

1 

1 .60000 

1 . 25000 

0 

1 .30000 

0 . 50000 

PSYCHE 

PHYSICAL 

PSYCH 

0.272727 

0 . 400000 

0 . 75000 

0.363636 

0 .000000 

0 .75000 

0 .000000 

0 .000000 

0 . 50000 

0 .000000 

0 .000000 

3 . 00000 

0.818182 

0 .000000 

0 .25000 

0.000000 

0 . 000000 

1 . 50000 

V 7 

va 

V 9 

4 

l 

0 

4 

0 

0 

0 

0 

0 

0 

0 

2 

0 

2 

0 

1 

1 

4 

V 1 6 

V 1 7 

via 

3 

4 

2 

4 

4 

2 

3 

3 

2 

3 

3 

3 

2 

4 

3 

4 

3 

1 

TECH 

INFORM 

SOLVE 

2 . 50  0 00 

1.66667 

2.00000 

1 . 50000 

2.44444 

1.0  8 3 3 3 

2 .00000 

0.77778 

1.9  1667 

2 . 00000 

0.88889 

1 . 3 33  3 3 

2 . 50000 

0.77778 

0 . 916  6 7 

2.50000 

1.22222 

1 . 91667 

EQUIP 

COMMUNIC 

DOMAIN 

0 . 66667 

1.23077 

1 

1 .16667 

0.92308 

1 

0.33333 

1 . 76923 

1 

0 . 66667 

3.07692 

1 

1.16667 

0 . 3Q769 

0 

0.83333 

1.23077 

1 

CLUSTER-4  

V4  V5  V6  V7  V 6 

2 3 0 2 0 

VI 3 VI 4 V 1 5 V 1 6 V17 

3 3 2 4 4 


V9 

2 

via 

3 


AIVAX$DUA1: { AI . 
TASK 

. KAT  AX 1 CLUSTER 3 
VI  9 

. LIS ; 1 

V20 

V21 

V22 

10- JUN- 
INPUTS 

1991  14:49 

COMPLEX 

TECH 

INFORM 

Page  5 
SOLVE 

forn  fil 

3 

3 

0 

2 

1 .20000 

0 . 500000 

2 

1.111X1 

1 .41667 

TASK 

RECALL 

PERCEPT 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAIN 

fora  £ 1 1 

2 

0 

0 

0 

0 

0 . 500000 

1 . 16667 

1 . 61538 

1 

— - 



Average  scores  for  eight  clusters 
CLUSTERS 

9 ; 

: 4 0 TUESDAY, 

APRIL  2,  1991  6 

TASK 

! VI 

V 2 

V 3 

V4 

V 5 

V 6 

V7 

V 8 

V 9 

medical 

0 

3 

4 

0 

3 

2 

0 

4 

2 

TASK 

vio 

Vll 

V 1 2 

VI  3 

VI  4 

V 1 5 

VI  fa 

VI  7 

via 

medical 

1 

2 

0 

2 

I 

4 

4 

2 

3 

TASK 

VI  9 

V20 

\ 

V 2 1 

V22 

INPUTS 

COMPLEX 

TECH 

INFORM 

SOLVE 

medical 

0 

2 

2 

2 

2 . 40000 

2 

2 

1.44444 

2.08333 

TASK 

RECALL 

PERCEPT 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAIN 

me  d 1 ca  1 

3 

0 

0 

0.909091 

0 

3 . 25000 

1.33333 

2.46154 

1 

— 

— 

CLUSTER=6  

— 

TASK 

VI 

V 2 

V 3 

V 4 

V5 

V 6 

V 7 

V 8 

V 9 

language 

0 

0 

0 

3 

3 

0 

0 

2 

2 

TAS  K 

V i 0 

Vll 

VI  2 

VI  3 

VI  4 

VI  5 

V 1 6 

VI  7 

v i a 

language 

1 

4 

1 

1 

2 

2 

4 

0 

4 

TASK 

VI  9 

V 2 0 

V 2 1 

V 2 2 

INPUTS 

COMPLEX 

TECH 

INFORM 

SOLVE 

language 

‘4 

3 

1 

0 

1 .40000 

S . 25000 

2 

0.888889 

2 . 25000 

TASK 

RECALL 

PERCEPT 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAIN 

language 

3.20000 

0 

0 

0 

0 

3 

0.333333 

3.38462 

1 

— 

— 

— 

CLUSTERS 

TASK 

VI 

V 2 

V3 

V 4 

V5 

V6 

V 7 

V 8 

V9 

a i r_t  raf 

0 

4 

4 

a 

1 

1 

0 

0 

0 

TASK 

VIO 

Vll 

VI  2 

V 1 3 

V 1 4 

V 1 5 

VI  6 

V 1 7 

via 

air_traf 

4 

4 

0 

2 

1 

0 

2 

0 

4 

TASK  ^ 

C 

V 1 9 

V 2 0 

V2  1 

v22  M 
c 

INPUTS 

j . 

COMPLEX 

TECH 

INFORM 

SOLVE 

iff 

\i 

f! 
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ai r^traf 

0 

0 

a 

0 

I .60000 

3 . 2 5000 

2 

1.22  2 2 2 

2 

TASK 

RECALL 

PERCEPT 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAIN 

air  tcaf 

2 . 4 0 0 fl  0 

4 

1.3  3 3 3 3 

2 . a 1 8 1 8 

0.800000 

2 . 2 5000 

1 . 3 33  3 3 

2 

2 

Average 

sco  res  for 

eight  clusters 

9:40 

TUESDAY,  APRIL 

2 , 1991  7 

— 

— 

CLUSTER 

= a 

TASK 

VI 

V 2 

V 3 

V4 

V5 

V6 

V 7 

V 8 

V 9 

weather 

1 

4 

4 

0 

0 

4 

2 

0 

3 

TASK 

V i n 

VI  1 

VI  2 

VI  3 

VI  4 

VI  5 

VI  6 

VI  7 

VI  8 

wea  t he  r 

0 

3 

0 

2 

0 

0 

4 

3 

0 

TASK 

VI  9 

V20 

V 2 1 

V 2 2 

INPUTS 

COMPLEX 

TECH 

INFORM 

SOLVE 

wea  the  r 

i 

3 

0 

0 

1 . 20000 

3.25000 

3 

1.66667 

2.3  3 3 3 3 

TASK 

RECALL. 

PERCEPT 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAIN 

wea  t he r 

2 . 60  00U 

1.50000 

0 

0 

0 

0 . 750000 

1 

0 

1 



CLUSTER 

= 9 

TASK 

VI 

V 2 

V 3 

V 4 

V 5 

V 6 

V 7 

V 8 

V 9 

con  a o le^ 

0 

0 

4 

0 

3 

2 

3 

4 

1 

TASK 

V J n 

Vll 

VI  2 

VI  3 

VI  4 

VI  5 

V 1 6 

VI  7 

VI  8 

console 

0 

1 

0 

2 

3 

0 

2 

4 

2 

TASK 

V 1 'J 

V 2 0 

V 2 1 

V 2 2 

INPUTS 

COMPLEX 

TECH 

INFORM 

SOLVE 

conso 1 e 

0 

0 

2 

0 

2.40000 

2 . 75000 

2 

1.88889 

2 . 8 33  33 

TASK 

RECALL 

PERCEPT 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAIN 

con  sole 

2 . 20000 

4 

1 

1.81818 

1.40000 

2 

1.66667 

1.84615 

2 

Ave  rage 

scores  for 

eight  clusters 

9:40 

TUESDAY,  APRIL 

2 , 1991  a 

CLUSTER 

VI 

V 2 

V 3 

V 4 

V 5 

V 6 

V 7 

V 8 

V 9 

1 

L.UUOOO 

0 . 50000 

4 .00000 

U . 00000 

3 . 50000 

2 . 50000 

1 . 50000 

0 . 00000 

2 . 50000 

2 

0 . 00000 

4.00000 

4.00000 

2.00000 

2 . 50000 

3 .00000 

0.00000 

3. 50000 

1 . 50000 

3 

0 . 16667 

0 .83333 

2 .66667 

0.33333 

2.16667 

0.33333 

1 . 50000 

0.66667 

1 . 00000 

4 

0 . 00000 

0 . 00000 

2 .00000 

2 . 00000 

3 .00000 

0 . 00000 

2 .00000 

0 .00000 

2 .00000 

5 

0 .00000 

3 . 00000 

4 .00000 

0 .00000 

3 .00000 

2.00000 

0.00000 

4.00000 

2 . 00000 

6 

0 .00000 

0 .00000 

0.00000 

3 . 00000 

3 .00000 

0 .00000 

0 .00000 

2.00000 

2 .00000 

7 

0 . 00000 

4 .00000 

4 .00000 

0.00000 

1 .00000 

1 .00000 

0.00000 

0 .00000 

0 .00000 

a 

1 . 00000 

4 .00000 

4.00000 

0 . 00000 

0 .00000 

4 . 00000 

2.00000 

0 .00000 

3 .00000 

9 

0 . 00000 

0 . 00000 

4.00000 

0.00000 

3 .00000 

2 . 00000 

3.00000 

4 .00000 

1 .00000 

AI VAX$ DUlAl : [ AX 

. KATAX 1 CLUSTER  3. 

. LIS ; 1 
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Page  7 

VlO 

Vll 

VI  2 

V 1 3 

VI  4 

Vl  5 

VI 6 

VI  7 

V 1 8 

VI  9 

2 .OOOOO 

4 

4.00000 

1 .00000 

0 

4 

4 .00000 

1 . 50000 

4 . 00000 

3 . 5Q000 

3 . 50000 

4 

0 .00000 

2 . 00000 

1 

0 

3 .50000 

4.00000 

3.50000 

0 . 00000 

0 . 50000 

3 

0 .66667 

2 .16667 

0 

1 

3 . 16667 

3.50000 

2 .16667 

1 . 16667 

2 .00000 

4 

3.00000 

3 .00000 

3 

2 

4 . 00000 

4 . 00000 

3 .00000 

3 . OOOOO 

l . 00000 

2 

0.00000 

2 .00000 

1 

4 

4 .00000 

2 .00000 

3 .00000 

0 . 00000 

1 .00000 

4 

1 .00000 

1 .00000 

2 

2 

4 . 00000 

0 . 00000 

4 .00000 

4 . 00000 

4 .000  0 0 

4 

0 .00000 

2 .00000 

1 

0 

2 . 00000 

0 . 00000 

4 .00000 

0 . 00000 

0 .00000 

3 

0 .00000 

2.00000 

0 

0 

4 . 00000 

3 . 00000 

0 .00000 

3 . OOOOO 

0 . 00000 

1 

0 .00000 

2 . 00000 

3 

0 

2 . 00000 

4 .00000 

2.000  00 

0 . OOOOO 

V 2 Q 

V 2 1 

V 2 2 

INPUTS 

COMPLEX 

^TECH 

INFORM 

SOLVE 

RECALL 

PERCEPT 

2 . 50000 

0.00000 

1 

1.30000 

0 . 75000 

1 .75000 

1.72222 

2 . 12500 

2.60000 

0 . 00000 

1 . 50000 

3 .00000 

1 

2 . 55000 

3 . 25000 

1 .75000 

0.72222 

2 . 70833 

2.60000 

2 . 50000 

0.66667 

0.83333 

1 

1 . 46667 

0.95833 

2 . 16667 

1 . 296  30 

1.61111 

2.10000 

0 . 00000 

3 . 00000 

0.00000 

2 

1.20000 

0 . 50000 

2.00000 

1.11111 

1.41667 

2.00000 

0.00000 

2 .00000 

2.00000 

2 

2.40000 

2 .00000 

2.00000 

1.44444 

2.08333 

3.00000 

0.00000 

3 .00000 

1.00000 

0 

1 .40000 

3 . 25000 

2 .00000 

0.88889 

2 .25000 

3.20000 

0.00000 

0 .00000 

0 .00000 

0 

1 . 60000 

3 . 25000 

2.00000 

1.22222 

2.00000 

2 . 40000 

4.00000 

3.00000 

0.00000 

0 

1 . 20000 

3 . 25000 

3 . 00000 

1 . 66667 

2 . 3 3 3 3 3 

2 .60000 

1 . 50000 

0 .00000 

2 .00000 

0 

2.40000 

2 . 75000 

2.00000 

1.88889 

2.83333 

2.20000 

4.00000 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMHUNIC 

DOMAIN 

0.00000 

O.UOOOO 

0 . 00000 

2 . OOUOO 

1 . 2500Q 

1.15385 

1.00000 

1.33333 

2.63636 

1.30000 

2 .62500 

1 .75000 

2.42306 

3.00000 

0 .41667 

0.24242 

0.06667 

1 .12500 

0 .80556 

1 .42308 

0.83333 

0 .00000 

0.00000 

0.00000 

0 . 50000 

1 .16667 

1 .61538 

1.00000 

0.00000 

0.90909 

0 .00000 

3 .25000 

1.33333 

2.46154 

1 .00000 

0 . 00000 

0 . 00000 

0.00000 

3 . 00000 

0.33333 

3 . 38462 

1 .00000 

1.33333 

2.81818 

0 . 80000 

2 . 25000 

1.33333 

2.00000 

2 . 00000 

0.00000 

0.00000 

0.00000 

0 . 75000 

1.00000 

0.00000 

1.00000 

1.00000 

1.81818 

1.40000 

2.00000 

1.66667 

1.84615 

2 . 0Q000 

SAS 


(! 

1 

1991 


( 


( 

14:25  THURSDAY,  JULi 


CENTROID  HIERARCHICAL  CLUSTER  ANALYSIS 


EIGENVALUES  OF  THE  COVARIANCE  MATRIX 


1 

2 

3 

4 

5 

6 
7 
a 

9 

10 
11 
12 
13 

1 4 

15 

16 
17 
1 8 

19 

20 

,21 

2 2 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 


EIGENVALUE 

13.8941 
7.6306 
5 .1732 
4.5940 
3.5219 
3.0940 
2 . 5 23  8 
1.7455 
1 .2473 
1.2109 
1 .0878 
0 . 9 389 
0.6308 
0 . 5047 
0.3128 
0.0000 
0 . 0000 
0 .0000 
0 . 0000 
0.0000 
0 . 0000 
0.0000 
0.0000 
0.0000 
0.0000 
0.0000 
0.0000 
-0 . 0000 
-0 .0000 
-0.0000 
-0 . 0000 
-0 .0000 
-0.0000 
-0.0000 
-0.0000 
-0.0000 


DIFFERENCE 

6.26352 
2.45736 
0 . 57925 
1.07205 
0 . 42791 
0 . 57024 
0.77827 
0.49820 
0.03612 
0 . 12  309 
0.11892 
0.30811 
0.12608 
0 . 19  18  4 
0 . 3 1 23  4 
0.00000 
0 . 00000 
0 . ooooo 
0 . ooooo 
0 . ooooo 
0.00000 
0.00000 
0.00000 
0.00000 
0 .00000 
0 .00000 
0 . ooooo 
0 .00000 
0 .00000 
0 . ooooo 
0 .00000 
0.00000 
0.00000 
0.00000 
0.00000 


PROPORTION 

0.288797 
0 .158606 
0 . 107529 
0.095489 
0.073205 
0.064311 
0.052458 
0 . 036  2 8 1 
0.025926 
0.025169 
0 .0226  10 
0.019515 
0.013111 
Q .010490 
0.006503 
0.000000 
0 . 0 0 0 0 0 0 
0 .00  0000 
0 .0000  00 
0.000000 
0 . 00000Q 
0.000000 
0 .00  0000 
0 . 000000 
0 . 000000 
0 .000000 
0 .000000 
- . 000000 
- .000000 
-.000000 
- .000000 
-.000000 
-.000000 
-.000000 
-.000000 
-.000000 


CUMULATIVE 

0.28880 

0.44740 

0.55493 

0.65042 

0.72363 

0.78794 

0.84039 

0.87668 

0.90260 

0 . 9 2777 
0.95038 
0.96990 
0.98301 
0.99350 
1.00000 
1.00000 
1.00000 

1 .00000 
1.00000 
1.00000 
J .ooooo 
1.00000 
1.00000 
1 . 0 0 0 0 0 
1 . ooooo 
1 . ooooo 
1 .00000 
1 .00000 
1 .00000 
1 .00000 
1.00000 
1.00000 
1.00000 
1.00000 
1.00000 
1.00000 


ROOT-MEAN-SQUARE  TOTAL— SAMPLE  STANDARD  DEVIATION  a 1.15603 

ROOT-MEAN-SQUARE  DISTANCE  BETWEEN  OBSERVATIONS  - 9.8092 


NUMBER 

OP 

CLUSTERS  CLUSTERS  JOINED 


FREQUENCY 
OP  NEW 
CLUSTER 


NORMALISED 

CENTROID 

DISTANCE 


15 

si#  desig 

protocol 

14 

pilot_t  r 

surgtry 

13 

cargo_lo 

account! 

12 

progra n_ 

lead#  rsh 

11 

•quip_di 

CL12 

2 0.578896 

2 0.622233 

2 0.666616 

2 0.671527 

3 0.663370 


10 
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principles 

procedures 

percep-speed 

search-rec-info 

id-ob-act-events 

scan-display 

cogattent 

attention  span 

training 

computer 
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notation 

communicates 


name:  drair_gen 

deductive 

acq-and-present 

databases 

historical 

calculate 

analyze 

compare 

facts 

answers 

transmits 


name:  air_traffic 

spacial 

temporal 

meansends 

sub-goals 

decompose 

visual 

verbal 

instrumentation 

mm-interface 

dynamism 

constraints 

estimate 

plan 

monitor 

procedures 

search-rec-info 

scan-display 

cogattent 

attention  span 

schedule 

training 

prob  of  success 

computer 

instruments 

communicates 

directs 


name:  pilot_training 

spacial 

visual 

kinesthetic 

instrumentation 

mm-interface 

branching 

dynamism 

simple-disc 

integrate 

plan 

monitor 
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interpret 

facts 

principles 

procedures 

search-rec-info 

scan-display 

confinement 

training 

vehicles 

instruments 


name:  equip_diag 

gen-and-test 

form-proc-alg 

ret-to-def 

visual 

written 

analyze 

deduce 

facts 

procedures 


name:  language_ train 

sub-goals 

decompose 

recombine 

verbal 

auditory 

written 

translate 

choose 

compare 

supervise 

facts 

education 

training 

articulation 

answers 

communicates 

instructs 


name:  form_fill  out 

acq-and-present~ 

form-proc-alg 

written 

propadeutics 

translate 

interpret 

answers 

transmits 


name:  sv_design 
temporal 
means  ends 


ll-JUL-1991  14:36 


Page 


_AIVAX$DUA1 s [AX. KATAX ] FOUR_LI ST . OUT ; 1 

sub-goals 

gen-and-test 

acq-and-present 

decompose 

recombine 

written 

propadeutics 

branching 

code 

computerize 

translate 

analyze 

integrate 

principles 

computer 


name:  cargo  loading 

mathematical 

form-proc-alg 

calculate 

compute 

estimate 

facts 


name:  leadership 

case-based 

sub-goals 

acq-and-present 

decompose 

verbal 

written 

itemize 

analyze 

articulation 

advises 

communicates 

negotiates 


name:  surgery 

spacial 

model-based 

means_ends 

sub-goals 

decompose 

visual 

instrumentation 
propadeutics 
complex-con t 
simple-disc 
fine 

categorize 

choose 

plan 

supervise 

facts 
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principles 

procedures 

search-rec-info 

id-ob-act-events 

contaminants 

preciseness 

cogattent 

attention  span 

education 

training 

instruments 

advises 

answers 

directs 

requests 

supervises 


name:  medical_diag 

temporal 

deductive 

cover-and-diff 

visual 

verbal 

written 

instrumentation 

historical 

branching 

categorize 

choose 

compare 

plan 

facts 

principles 

procedures 

id-ob-act-events 

education 

training 

articulation 

advises 

answers 

communicates 

directs 

informs 


name:  accounting 

mathematical 

sub-goals 

form-proc-alg 

decompose 

written 

databases 

categorize 

calculate 

i temize 

tabulate 

compare 
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compute 

procedures 

education 

notation 


name:  protocol_des 

statistical 

means_ends 

sub-goals 

written 

branching 

translate 

plan 

principles 

cases-examples 

education 
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10:29  THURSDAY,  JULY  11 , 


1991  1 

INITIAL  FACTOR  METHOD:  PRINCIPAL  COMPONENTS 


PRIOR  COMMUNALITY  ESTIMATES: 


EIGENVALUES  OF  THE 

CORRELATION  MATRIX: 

TOTAL  = 

36 

AVERAGE 

1 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

EIGENVALUE 

9.617646 

4.988310 

4 .711803 

3 .693210 

2.925432 

2.417242 

1 . 520797 

1.371233 

1.091349 

0.899565 

0.8610 62 

0 

673311 

DIFFERENCE 

4.629336 

0.276507 

1.018593 

0 . 767778 

0 . 508190 

0.896445 

0.149564 

0.279884 

0.191784 

0.038503 

0.187750 

0 

116727 

PROPORTION 

0 . 2672 

0.1386 

0 . 1309 

0.1026 

0.0813 

0.0671 

0.0  4 22 

0.0381 

0.0  30  3 

0.0250 

0.0  2 39 

0.0187 

CUMULATIVE 

0.2672 

0.4057 

0.5366 

0.6392 

0.7205 

0.7876 

0 . 8 298 

0.0  6 79 

0.898  3 

0.9  2 3 2 

0.9472 

0.9659 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

EIGENVALUE 

0.556585 

0.436890 

0 . 235567 

0 . 000000 

0.000000 

0.000000 

0.000000 

0 . 00  0000 

0.000000 

0 . 000000 

0.000000 

0 

000000 

DIFFERENCE 

0.119695 

0.201323 

0 . 235567 

0 . 000000 

0 .000000 

0 . 000000 

0 . 000000 

0.000000 

0 . 000000 

0 . 000000 

0 . 000000 

0 

000000 

PROPORTION 

0 0155 

0.0  121 

0 . 0065 

0.0000 

0.0000 

0 . 0000 

0 . 0000 

0 . 0000 

0.0000 

0.0  000 

0.0000 

0 . 0000 
CUMULATIVE 

0.9813 

0.99  35 

1 . 0000 

1 .0000 

1 .0000 

l . 0000 

l . oooo 

1 .000  0 

l . 0000 

1.0000 

l .0000 

1.0000 

25 

26 

27 

28 

*0 

30 

3 1 

32 

3 3 

M 

35 

36 

EIGENVALUE 

0.000000 

0.000000 

0.000000 

0 . 00  0000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0 . 000000 

0 

000000 

DIFFERENCE 

PROPORTION 

0 . 000000 
0 .0000 

0 .000000 
0 .0000 

0.000000 
0 .0000 

0 . 000000 
0 .0000 

0 . 000000 
0.0000 

0.000000 
0 .0000 

0 .000000 
0 . 0000 

0.000000 

0.0000 

0 .000000 
0.0000 

0 . 000000 
0.0000 

0 . 000000 
0.0000 

0.0000 

CUMULATIVE 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1.0000 

1 .0000 

1.0000 

1.0  000 

1.0000 

1.0000 

3 FACTORS  WILL  HE  RETAINED  BY  THE  N FACTOR  CRITERION 

SAS 


10:29  THURSDAY.  JULY  11 


INITIAL  FACTOR  METHOD:  PRINCIPAL  COMPONENTS 


FACTOR  PATTERN 
FACTOR!  FACTOR l FACTOR. 


VI 

-0.10992 

-0.37229 

0.19889 

V2 

0 . 79963 

0 .18055 

-0 .47424 

V3 

0.43339 

0.24648 

0 .37934 

V 4 

-0.04173 

0.37413 

0.49575 

V5 

-0.32641 

0.36362 

0.51651 

V6 

0.76174 

-0.25723 

-0.03079 

V7 

-0.46743 

-0.41143 

-0 .31061 

PRECEDING.  PAGE  BLANK  NOT  FILMED 


V8 

V9 

V10 

Vll 

V12 

V13 

VI  4 

VI 5 

V16 

V17 

via 

VI  9 

V20 

V21 

V22 

INPUTS 

COMPLEX 

TECH 

INFORM 

SOLVE 

RECALL 

PERCEPT 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUN I C 

DOMAIN 


0.11742 
-0.10878 
—0 .06887 
-0.30244 
-0 . 3 5806 
0 .27021 
0.56944 
-0 .55791 
-0 . 56262 
0 .11202 
0.01511 
-0 .28547 
-0.31312 
0 .22520 
0.09533 
0 .73434 
0.88211 
0 .31606 
-0  . 26132 
0 .75982 
0 . 75994 
0 . 89607 
0.66407 
0.81357 
0.77210 
0.65707 
0 . 71017 
0 . 51370 
-0.04690 


-0 . 76308 
0.52321 
0.79410 
0.69378 
-0 .22827 
-0.75703 
-0.33047 
0.27648 
-0 .13935 
-0 .22004 
0.44216 
0.12366 
0.30821 
-0.03442 
-0 .45801 
-0 . 12936 
0.13318 
-0  .11731 
-0.34782 
0.00519 
-0 . 00247 
0 .03397 
0 .01200 
0.29907 
0.33064 
0.29248 
-0.12855 
0 . 30002 
0.56915 


0.29772 
0.34057 
-0.27549 
-0.30752 
0 .37920 
0.04142 
0.71624 
0 .05587 
-0 . 06938 
-0 . 52748 
-0 .33579 
0 . 05299 
0 . 43738 
0.12403 
0 . 35984 
0 .09583 
0 . 17860 
0 . 30677 
-0 .06739 
0 . 28989 
0.41733 
-0  . 23900 
-0 . 44085 
-0.34481 
-0.44444 
0.45892 
-0  . 28133 
0 . 46156 
0.67331 


VARIANCE  EXPLAINED  BY  EACH  FACTOR 

FACTOR 1 FACTGR2  FACTOR3 
9.617646  4,988310  4.711803 


FINAL  COMMUNALITY  ESTIMATES:  TOTAL  = 19.317759 


VI 

V 2 

V 3 

V 4 

V5 

V 6 

V 7 

V8 

V9 

V10 

VI 1 

VI 

0.190240 

0.896920 

0 . 392475 

0.387490 

0 . 505542 

0 ,647356 

0.484246 

0 .684708 

0 .401139 

0.711212 

0.667376 

0.3  24  1 0 

X 

713 

VI  4 

V 1 5 

VI  6 

VI  7 

VI  8 

V l 9 

V 2 0 

V 2 1 

V 2 2 

INPUTS 

COMPLE 

7 

1 

0 . 6 4 7S  22 

0.946478 

0.390830 

0.340772 

0.339201 

0.308487 

0.049593 

0.384339 

0 . 0 b 7 2 8 2 

0 . 348 349 

0.565171 

0 . 8 2 7 75 

3 

SAS 

10:29 

THURSDAY, 

JULY  11. 

19  91 

INITIAL 

FACTOR  METHOD:  PRINCIPAL  COMPONENTS 

N 

TECH 

INFORM 

SOLVE 

RECALL 

PERCEPT 

ENVIRON 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAI 

0 . 207762 

0.193813 

0 .661386 

0 . 751682 

0 .861208 

0.635477 

0.870237 

0 . 902986 

0 . 727900 

0 .600010 

0 . 566939 

0.77947 

3 


1 SAS  10:29  THURSDAY , JULY  11. 

1991  4 


(, 

ROTATION  METHOD:  VARIMAX 


1 

1991  5 


( 


ORTHOGONAL  TRANSFORMATION  MATRIX 
1 2 3 


1 

0 .90252 

-0  .43052 

0.01021 

2 

0 . 20344 

0 . 44714 

0.87102 

3 

0 . 37955 

0 . 78404 

-0 .49114 

ROTATED  FACTOR  PATTERN 

FACTORI 

FACTOR2 

F ACTOR  3 

VI 

-0 .09945 

0.036  80 

-0.42308 

V 2 

0 . 57842 

-0.63535 

0.39835 

V 3 

0.58527 

0.22104 

0.03280 

V4 

0 . 22661 

0 . 57395 

0.08197 

V5 

-0 .02458 

0 . 70808 

0.05971 

V6 

0 .62347 

-0 . 46710 

-0 .20116 

V7 

-0.62346 

-0.22626 

-0.21058 

V8 

0 . 06373 

-0.15833 

-0 .8096  8 

V9 

0.13934 

0.54694 

0.28737 

V10 

-0.00499 

0.16  86  4 

0 .8  26  29 

Vll 

-0 .24854 

0 . 19932 

0.75225 

VI  2 

-0 . 22567 

0.349  3 9 

-0.38872 

V 1 3 

0.10558 

-0.42235 

-0.67698 

V 1 4 

0.71855 

0.16864 

-0.63381 

V 1 5 

-0.42608 

0.40762 

0 . 20768 

V 1 6 

-0.56246 

0 .12551 

-0.09304 

V 1 7 

-0 .14387 

-0 . 56018 

0 .06855 

VI  8 

-0.02386 

-0 .07207 

0 . 55020 

VI  9 

-0.21238 

0.21974 

0 .07877 

V 2 0 

-0.05389 

0.61554 

0 .05045 

V 2 1 

0.2433  2 

-0.01510 

-0 .08859 

V 2 2 

0.12944 

0 .03630 

-0.57470 

INPUTS 

0.67281 

-0 . 29885 

-0 .15225 

COMPLEX 

0.89101 

-0 .18018 

0.03729 

TECH 

0.37782 

0 .05200 

-0 . 24962 

INFORM 

-0.33219 

-0 .09586 

-0 . 27253 

SOLVE 

0 . 79684 

-0 .09751 

-0  .13010 

RECALL 

0 . 84376 

-0 .00107 

-0  . 19937 

PERCEPT 

0.72492 

-0 .55797 

0.15611 

ENVIRON 

0.43446 

-0.62617 

0 . 23375 

PSYCHE 

0 . 66424 

-0.48688 

0.43815 

PHYSICAL 

0.59542 

-0.53302 

0.51415 

PSYCH 

0 . 82671 

0 . 20771 

0.03607 

EQUIP 

0 . 50801 

-0.58379 

0.03345 

COMMUNIC 

0.69985 

0.27488 

0.03987 

DOMAIN 

0.32902 

0.80258 

0.16457 

VARIANCE  EXPLAINED  BY  EACH  FACTOR 

FACTORI  FACTOR2  FACTOR3 
8. 719305  5.676360  4.922094 

SAS 


10:29  THURSDAY,  JULY  11 


ROTATION  METHOD:  VAR I MAX 


FINAL  COMMONALITY  ESTIMATES:  TOTAL  = 19.317759 


VI 

V2 

V 3 

V4 

V5 

V6 

0 .190240 

0 .896920 

0 .392475 

0 . 387490 

0 .505542 

0.647356 

V 1 3 

VI  4 

V 1 5 

VI  6 

V 1 7 

via 

0.647822 

0.946478 

0.390830 

0 . 340772 

0 . 339201 

0 . 308487 

TECH 

INFORM 

SOLVE 

RECALL 

PERCEPT 

ENVIRON 

0 . 21 0 7 7 6 2 

0.193813 

0.661386 

0 . 751682 

0.861208 

0.635477 

V7 

V8 

V9 

V10 

Vll 

VI 

0 .484246 

0.684708 

0.401139 

0 .711212 

0 .667376 

0 .32410 

V 1 9 

V 2 0 

V21 

V 2 2 

INPUTS 

COMPLE 

0 .099593 

0 . 384339 

0 .067282 

0.348349 

0 . 565171 

0 .82775 

PSYCHE 

PHYSICAL 

PSYCH 

EQUIP 

COMMUNIC 

DOMAI 

0.870237 

0.902986 

0.727900 

0.600010 

0.566939 

0.77947 

v 


8 CLUSTER  SOLUTION 


1 

1991  1 


SAS 


12:48  THURSDAY,  JULY  11 


CENTROID  HIERARCHICAL  CLUSTER  ANALYSIS 
EIGENVALUES  OF  THE  COVARIANCE  MATRIX 


EIGENVALUE 

DIFFERENCE 

PROPORTION 

CUMULATIVE 

1 

10.6128 

2.60119 

0.246740 

0 . 2 4 b 7 4 

2 

8.0116 

2.13421 

0.186264 

0.43300 

3 

5 . 8774 

1.84754 

0 .136645 

0. 56965 

4 

4.0299 

0.70215 

0 .093691 

0.66334 

5 

3 .3277 

0 . 27633 

0 .077367 

0.74071 

e 

3.0514 

1 . 35031 

0 .070942 

0.81165 

7 

1.7011 

0 . 18551 

0.039549 

0.85120 

8 

1.5156 

0 . 29187 

0.035236 

0.88643 

9 

1.2237 

0 .17295 

0.028450 

0 . 9L488 

10 

1.0507 

0. 182  34 

0.024429 

0.93931 

11 

0.8684 

0 . 22584 

0.020190 

0.95950 

12 

0.6426 

0 . 12845 

0.014939 

0.97444 

13 

0.5141 

0.13462 

0 . 011953 

0.98639 

14 

0.3795 

0 . 17373 

0.008823 

0.99522 

15 

0.2058 

0.2057b 

0.004784 

1.00000 

16 

0 . 0000 

0.00000 

0 . 000000 

1.00000 

17 

0 . 0 00  0 

0.00000 

0 .000000 

L . 00000 

18 

0.0000 

0.00000! 

0 .000000 

t . 0000  0 

19 

0.0000 

o.ooooo. 

0 . 000000 

1 . 00000 

20 

0.0000 

0.00000 

0 . 000000 

1.00000 

21 

0 .0000 

0.00000 

0 . 000000 

1.00000 

22 

0 .0000 

0 . 00000 

0.000000 

1.00000 

23 

0.0000 

0.00000 

0 .000000 

1.00000 

24 

0.0000 

0 . 00000 

0.000000 

1.00000 

25 

0 .0000 

0 . 00000 

0 . 000000 

1 . 00000 

26 

0.0000 

0 .00000 

0 .000000 

1.00000 

27 

-0 .0000 

0 .0  000  0 

- . 000000 

1.00000 

28 

-0.0000 

0 . 00000 

- .000000 

1.00000 

29 

-0 .0000 

0 . 00000 

-.000000 

i .00000 

30 

-0.0000 

0 . 0 0 0 0 0 

- . 000000 

1 . oocoo 

31 

-0.0000 

0 .00000 

- . 000000 

1.00000 

32 

-0.0000 

0 . 00000 

- . 000000 

1,00000 

33 

-0.0000 

0 , 00000 

- , 000000 

1.00000 

34 

-0.0000 

0 .00000 

- .000000 

1 .00000 

35 

-0.0000 

0.00000 

- . 000000 

1.00000 

36 

-0  . 0000 

- . 000000 

1.00000 

ROOT-MEAN -SQUARE 

TOTAL-S AMPLE  STANDARD  DEVIATION 

- 1 . 09  3U0 

ROOT-MEAN- SQUARE 

DISTANCE  BETWEEN 

OBSERVATIONS 

* 9.27493 

NUMBER 

[•  REQUENCY 

NORMALIZE  I'« 

OF 

OF  NEW 

CENTROID 

CLUSTERS  CLUSTERS 

JOINED 

CLUSTER 

DISTANCE 

15  s w das i g 

leadersh 

2 

0.564206 

14  air  traf 

surgery 

2 

0 .610276 

13  CL14 

pilottr 

3 

0 . 582182 

12  cargo_lo 

accounti 

2 

0 .623189 

11  progra*_ 

f ° r*_f i 1 

2 

0 ,672973 

|l  I HUH  I 


£ H 


0 


1991  2 


1 

1991 


D 

I 

S 

T 

A 

N 

C 

E 

8 

E 


E 

E 

N 

C 

L 

U 

s 


10 

;;N 

CL  1 1 

G 

CLl  2 

a 

0 .677967 

( 

9 

CLIO 

dr ai r ge 

5 

0.658739 

8 

console^ 

aadica  1 

2 

0.698351 

SAS 

12:48  THURSDAY, 

JULY 

CENTROID 

HIERARCHICAL 

CLUSTER  ANALYSIS 

NUMBER 

FREQUENCY 

NORMALIZED 

OF 

OF  NEW 

CENTROID 

CLUSTERS 

CLUSTERS 

JOINED 

CLUSTER 

DISTANCE 

7 

CL9 

C LI  5 

7 

0 . 70  5751 

6 

CL7 

p r o toco  1 

3 

0 .73  3092 

5 

we  a the  r 

CL8 

3 

0.749222 

4 

C L 5 

equ i p d i 

4 

0 . 78  74  31 

3 

CL6 

CL4 

12 

0.739529 

2 

CL3 

CLl  3 

15 

0.762808 

1 

CL2 

language 

16 

0.787522 

SAS 

12:48  THURSDAY, 

JULY 

CENTROID 

HIERARCHICAL 

CLUSTER  ANALYSIS 

NAME  OF  OBSERVATION  OR 

CLUSTER 

p 

f 

c 

a d 

s 

1 

P 

c re  e 

P 

1 

r 

o 

a 

c r 

w 

e 

1 w 

o e q 

i s i 

a 

o 

r 

r 

c a 

a 

o e 

n d u 

r u 1 

n 

g 

re 

g 

O 1 

d 

d 

c a 

s i i 

r o 

g 

r 

o 

u r 

e 

e 

o t 

o c p 

t g t 

u 

a 

f 

n 

s 

r 

c h 

1 a 

i e 

a 

m 

i 

I 

t g 

l 

5 

o e 

eld 

art 

0 

1 

o 

i e 

g 

h 

1 r 

l 

f y r 

e 

0.8  + 
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0.6  + 
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T 
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E 
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R 
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I 

C 
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1 ■ 

E 

N 

0.2  + . 

T 
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R 

1 • 

O 

I 

1 - 

D 

0.1  + . 

S 

1 - 
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1 . 
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1 

1 

1991  4 

Average  scores  for 

eight  clusters 

12 
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1 1 , 
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- CLUST  E R = 1 

— 



— 

— 

V9 

TAS  K 

VI 

V 2 

VI 

V 4 

V5 

V 6 

V 7 

VS 

0 

sw  de s i g 

0 

0 

4 

0 

i 

2 

2 

0 

0 

leader  ah 
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APPENDIX  0 

FINAL  LIST  OF  CHARACTERISTICS 

1.  Representational  Techniques  - characteristics  related  to  the  use  of 

information  to  perform  the  task 


statistical 

spatial 

temporal 

analogical 

case-based 

modelling 

mathematical 

2.  Search  Strategies  - methods  of  finding  a solution  in  the  problem  space 

means -ends  analysis 

forming  sub-goals 

gene rate -and- test 

goal-directed  heuristic  search 

data-drive  heuristic  search 

cover  & differentiate 

propose  & refine 

acquire  & present 

formula/procedure/algorithm 

decomposing 

recombining 

generalizing 

specialization 

return  to  definition 

3.  Inputs  - information  input  to  the  task 

qualitative 
quantitative 
visual 
verbal 
auditory 
kinesthetic 
• written  material 
sensor  readings / test  results 
interface  to  machine/console/device 
databases 

4.  Task  Complexity  - characteristics  of  the  data  array  and  problem  solving  space 

that  influence  the  task 


branching  factor 
dynamism 

physical  constraints 
time  constraints 
uncertainity  in  inputs 

uncertainity  in  interpretation/evaluation 


5.  Technical  Component  - the  degree  to  which  the  task  requires  a technical 

background  or  skill 


technical /engineering 
non-technical 


6.  Motor  Processes  - physical  manipulation  during  the  task 

Complex-continuous 

compound 

reflex 

simple -discrete 

fine 

gross 

repetitive 

7.  Information  Processing  - characteristics  of  the  way  data  is  manipulated 

categorize 

calculate 

code 

computerize 

interpolate 

itemize 

tabulate 

translate 

analyze 

deduce 

induce 

choose 

compare 

compute 

estimate 

integrate 

monitor 

interpret 

8.  Recall  - the  role  of  memory  in  the  task 
facts 

principles 

procedures 

analogs  7 - 

cases 

models 


9.  Perceptual  - use  of  the  visual  input  during  task  performance 
perceptual  speed 

searching  for  and  receiving  information 
identifying  objects,  actions,  and  events 
scanning  a display 

10.  Environment  - physical  and  psychological  characteristics  of  the  environment 

in  which  the  task  is  performed 


10.1  Physical 

acceleration 

confinement 

isolation 

contaminants 

lighting 

magnetism 

noise 

10.2  Psychological 

fatigue 
mental  strain 
stress 

physical  strain 
precision 

attention /automatic ity 
response  chaining 
amount  of  sleep 
work  schedule 
boredom 

11.  Physical  Characteristics  - characteristics  of  the  problem  solver 
age 

gender 
. height 
weight 

12.  Background  - type  of  learning  situation  encountered  by  problem  solver 

education 

training 


13.  Type  of  Domain 

knowledge -rich 
high  performance 


14.  Hardware /Equipment /Tools  Involved  - items  or  devices  used  during  the  task 


computer 

vehicles /aircraft 

weapon  system 

instruments 

notation 

tools 

test  equipment 


15.  Communication  Processes  - communication  during  task  performance 

15.1  Verbal 

negotiates 

supervises 

transmits 

requests 

instructs 

informs 

indicates 

directs 

communicates 

answers 

advises 

15.2  Nonverbal 

interpretive  movement 
expressive  movement 


Table  19 

Human's  taxonomy 


l Environment 

1.1  Type 

1.1.1  Laboratory 

1.1.2  Office 

1.1.3  Outer  space 

1.2  Attributes 

1.2  I Acceleration 
122  Confinement 

1.2.3  Contaminants/ toxicants 

1.2.4  Day/night  cycles 

1.2.5  Electricity 

1.2.6  Isolation 

1.2.7  Lighting 

1. 2.7.1  Type 

1.2.7. 1.1  Fluorescent 

1.2. 7. 1.2  Incandescent 
1.2.7.  L3  Sunlight 

1.2. 7. 2 Attributes 

1.2. 7. 2.1  Luminance  in  foot  lamberts 


Magnetism 

Noise 
1.2.9. 1 

Duration 
1.2. 9. 1.1 

Continuous 

1.2.91. 2 

Impulsive 

1.2.91. 3 

Intermittent 

1.2.9. 1.4 

Single 

12.9.2 

Frequency 

1.2.9.2.1 

Constant 

1.2. 9.2. 2 

Variable 

1.2.93 

Intensity 
1.2. 9.3.1 

Constant 

l. 2.9.3. 2 

Variable 

1.2  9.4 

Medium 

12.9.4  1 

Aunoapbere 

1.2.9. 4. 2 

Communication 

1 2 9 4.3 

Hydrosphere 

1.2. 9. 5 

Range 
1 2.9.5  I 

Infrasonic 

l 2 9.5.2 

Sonic 

1 2.9  5 3 

Ultrasonic 

1.2. 9. 6 

Spectrum 
1 .2.9.6. 1 

Broad  band 

1 2.96.2 

Narrow  band 

1 2.9  6.3 

Pure 

1.2.10  Pressure 

1.2.10.1  Ambient  vapor  pressure  in  MB 

1.2.10.2  Gravity 

1.2.11  Radiation 

1.2.11.1  infrared 

1.2.11.2  Microwave 
L2.11.3  Radio  frequency 

1.2.1 1.4  Ultraviolet 

1.2.11.5  Visible 

1.2.11.6  X-Ray 

1.2.12  Reduced/zero  gravity 

1.2.13  Terrain 

1.2.14  Thermal 

1.2.14.1  Ambient  dry  bulb  temperature  in  degrees  Celsius 

1.2.14.2  Humidity 

1.2.15  Vibration 


n '“hv**  cta..c.en».» 
211  Age 
i I 2 Effector 

‘ 2121  2T2.ll 

2 1212 
2.12.1 .3 


1 1 > 2 Hands 

2.1. 2.21 
2.12.2.2 
2.12.2.3 


Agility 

Dominance 

Lift  strength 

Dominance 

Flexibility 
Grip  strength 


2.1.3 

2.1.4 


2.1.5 

2.16 


2.12.3  Voice 

Fatigue 

Gender 

2 1.4  1 Female 
2. 1.4. 2 Male 
Height  in  cin 
Limbs 

2.1.6  i-2 


Endurance 

Strength 


2162  ES.2.1 

2 1.7  Weight  in  Kg 
2 2 Mental  state 

2.2.1  Attention  span 
22-2  Drugs  ^ - 
2 2.2.1  Type 
■>•>2  2 Attributes 
2 2.22.2.1 
22.2.2.2 
2.2.2.2.3 


Length 


KK.oio.y‘»''“U!‘ukt" 

Number  of  days  taken 


2.2.3 


Memory 

2.2.3.1 

2. 2. 3.2 


Number  o.  Um»  »“  “““  “*  M,°” 

S-H" Number  oUemssroreri 


2.2.4 

2.2.5 

2.2.6 


if /'pVrS'.veri  proMbrlrr,  ol  sure... 
ep  in  hour* 

*6  r^ays  on  duty 

2 2.6.2.2  Fr~l— 


2.3 


Senses 

2.3.1 


2.3.2 

2.3.3 

2.3.4 


Auditory 
2.3.11  Acuity 
2 3.12  Biaural 
>113  Monaural 

2 3.14  Tone  percepts 

Olfactory 

Tactual 


Vision 

2.3.4. 1 
2.3.4  2 

2.3.4  3 

2. 3.4.4 

2.3.45 

2.3.46 


Accommodation 

Acuity 

Binocular 
Color  perception 

Convergence 

Monocular 


} 1 jik 

/ } i Control  device 

3l.l  Type 
3 111 
3 1 12 
3 113 
3 1.1.4 
3 1.15 


3 11.6 
3 1.1.7 


Knob 

Lever 

Pedal 

Pushbutton 

Switch 

3.1.1.5  1 Rocker 

3115  2 Rotary  selector 
3.1.15.3  Toggle 
Track  ball 
Touch  device 
3 1 1.7.1  Keyboard 

3 1.1.7  11  Membrane 
3.1.1.7.12  Teletype 
Light  pen 
Pointer 
Touch  panel 
Touch  screen 


3. 1.1.8 

3 1 1.9 


3.1. 1.7.2 
3. 117. 3 

3.1.17.4 

3. 1.17. 5 
Voice  activated 
Wheels 

3 11.9.1  Steering  wheels 
3.1.1.9  2 Thumbwheels 


3.1.2 


Attributes 

3. 1.2.1  Number  of  positions 
Size. 

Type  damping 
Type  feedback 


3.2 


3 1.2.2 
3. 1.2.3 
3.1  2.4 
Display  device 
3.2.1  Type 
3 2.1.1 


Auditory  displays 
3 2 111  Electromechanical 
3.2.1. 1.1.1 
3.2.1. 11.2 
3 2.1.11.3 
3.2.1. 11  4 
Electronic 
3.2.1. 1.2.1 
3.2.1. 1.2.2 


Bells 

Buzzers 

Horns 

Sirens 


3.2.1. 1.2 


Electronic  tones  and  signals 
Recorded  signals  directions 


3.2. 1.2 


Visual 
3. 2. 1-2.1 


3.2. 1.2.2 


3.2. 1.2. 3 


3.2. 1.2. 4 


3.2.12.5 


CRT  alphanumenc-pictonal  displays 
3 .2.1.2.11  Computer  output  displays 
3 2 12  12  Infrared  sensor  displays 
3 2 12  1.3  Low-light-tevel  TV  displays 
3 2 12  14  Television  output  displays 

3 2 1 2.2.2  Bargraph  displays 
32  1.2.2.3  Waveform  displays 
CRT  spatial  relation  displays 
3 2.1 .2.3.1  Radar  displays 

3.2.12.3.2  Sonar  displays 
Hard  copy  readout  displays 
3 2.1.2.4.1  Plotters 

3 21.2.4.2  Printers 
3 2 1 2 4 3 Recorders 

3.2.1. 2.5.2  Multiple  status 
3.2.12.5.3  Single  status 


^ 2 1.2.6  l ighl  Minilling  Diode  (LED) 

3 2. 1.2.7  l iquid  Crystal  Displays  (LCD) 

3 2.1  2.8  Mechanical 

3 2 1 2 9 Projection 

3 2.1.2.10  Random-access  digital  readouts 

3 2 12  10.1  Back-lighted  belt  displays 
3 2.1.2.10.2  Cold  cathode  tubes 

3.2. 1.2. 10.3  Edge-lighted  plates 
3 2 1.2  10  4 Light-Emitting  Diode  displays 
3 2.1  2.10.5  Projection  readouts 

3 2 L2.10.6  Segmented  matrices 

3 2.1.2.1!  Scalar  displays 

3.2. 1.2.1 1.1  Fixed  pointer,  moving  scale 
3 2.L2.1L2  Moving  pointer,  fixed  scale 

3 2.1  2.12  Sequential-access  digital  readouts 

3.2. 1 .2. 12.1  Electromechanical  drum  counters 
3 2.1.2.12.2  Flag  counters 

3.2.1.2.13  Status  displays 

3. 2. 1.2. 13. 1 Large  screen  displays 
3 2.1.213.2  Map  displays 

3.2  1.2.13.3  Matrix  boards 
3 2.1.2.13.4  Plot  boards 
3 2.1.2.13.5  Projected  displays 

3 2.2  Attributes 

3.2.2. 1 Size 

3.2.2. 1.1  Diameter  in  cm 
3 2.2. 1.2  Height  tn  cm 
3 2.2. 1.3  Width  in  cm 

3. 2. 2. 2 Viewing  conditions 

3 2.2.2. 1 Collimanon 

3 2.2. 2.2  Distance  of  operator  to  display  in  cm 
3 2.2.2. 3 Magnification 
3 2. 2.2. 4 Ocular  design 

3.2.2.24. 1  Binocular 
3 2 2.2.4  2 Dichoptic 

3.2.2.2.4.3  Monocular  left  eye 

3. 2. 2.2.4. 4 Monocular  right  eye 
3 2.2.2. 5 Resolution 

3 2.2.2.6  Visual  angle/field  of  view  in  degrees 

3.3  Machine 

3.3.1  Computer 

3. 3.1.1  Mainframe 

3.3. 1.2  Personal 

3.3.2  Vehicles 

33.2.1  Aircraft 

3.3.2.  Li  Helicopter 
3.3.2.12  Jet_Z“  _ 

3 3.2.1.3  Propeller 

3. 3.2.2  Motorized  ground  vehicle 

3. 3. 2.2.1  Car 

3.3.2.2  2 Half  track 
3 3.2.2.3  Jeep 

3.32.2.4  Tank 

3 3.2.2.5  Truck 
3.3.23  Ship 

3. 3.2.3. 1 Aircraft  carrier 

3.32.3.2  Destroyer 

3.32.3. 3 Submarine 

3. 3.2. 4  Spacecraft 

3.3.3  Weapon 


3.4  Stimulus 

3.4.1  Type 

3. 4. 1.1  Auditory 

3 4 1.2  Kinesthetic 

3.4. 1.3  Visual 

3 4.1.3.1  Alphanumeric 

34.13.2  Graph 

3.4.2  Attributes 

3.4.2. 1 Background 

3. 4. 2. 1.1  Complexity 

3 4.2.1  2 Contrail 

3.4.2. 1.3  Number  of  background  characters 

3. 4.2.2  Characteristics 

3 4.2.2.1  Alphanumerics 
3 4. 2.2. 2 Changing/moving  stimulus 

3.4  2.2.3  Coded  stimulus 
3.4.2.24  Conspicuity 

3. 4. 2. 2. 5 Raw  stimulus 

3. 4. 2. 2. 6 Static  stimulus 
3 4 2.3  Color 

3. 4. 2. 4 Duration 

3. 4. 2. 4. 1 Continuous 

3. 4. 2. 4. 2 Intermittent 

3. 4. 2.4.2. 1 Probability 
3.4.2.4.22  Rate 

3.42.4.3  Single 

3. 4. 2. 5 Information  presented 

3.4.25. 1 Content 

3. 4. 2.5. 2 Qualitative 

3.42.5.3  Quantitative 

3. 4.2. 6 Location  on  display 

3. 4. 2.6.1  Center 

3 4.2.6.2  Lower  left 

3. 4. 2. 6. 3 Lower  middle 

3. 4. 2. 6. 4 Lower  right 

3.4.2. 6.5  Middle  left 

3 4 2.6.6  Middle  right 
3.4  2.6.7  Upper  left 

3. 4. 2.6. 8 Upper  middle 

3. 4. 2. 6. 9 Upper  right 

3.4.2.6.10  Predictability  of  location 

3. 4. 2.7  Mechanism 

3. 4. 2. 7. 1 Directly  viewed  event 
3 4 2.7.2  Display 

3. 4. 2. 7. 3  Written  material 

3. 4. 2.8  Number 

3.42.8.1  Multiple 

3.42.8.2  Single 

3. 4. 2. 9 Range  of  values 

3.4.2.10  Relative  movement 

3.4.2.10.1  Observer  and  target  at  rest 

3.4.2. 10.2  Observer  and  target  in  motion 

3.4.2.10.3  Observer  in  motion,  target  at  rest 

3.4.2.10.4  Observer  at  rest,  target  in  motion 

3.4.2. 1 1 Relative  position  of  observer 

3. 4. 2.1  LI  Horizontal  range  in  Km 

3.4.2.11.2  Offset  in  Km 

3.4.2.11.3  Positions 

3.4.2.11.3.1  Air-to-air 

3.4.2.11.3.2  Air-to-ground 

3.4  2. 11.3.3  At  a display 

3.4.2.11.3.4  Ground-to-air 

3.4.2.11.3.5  Ground-to-ground 

3.4.2.12  Size/amplitude 

3.4.2.12.1  Signal-to-noise  ratio 


3.5  Task 

35.1  Type 

3. 5. 1.1  Communication 

35.1.1.1  Type 

35.1.1. 1.1  Advise 

3.5. 1.1. 1.2  Answer 

3. 5. 1.1. 1.3  Communicate 

3.5. 1.1. 1.3.1  Job-related 

3.5. 1 .1. 1.3.2  Public- related 
3.5.1.1.14  Comprehend 

3. 5. 1.1. 1.5  Coordinate 

3.5. 1.1. 1.6  Direct 

3.5. 1.1. 1.7  Indicate 

3. 5. 1.1. 1.8  Inform 

3. 5. 1.1. 1.9  Instruct 
3 5. 1.1. 1. 10  Request 

3.5.1.1.1.11  Supervise 

3.5.1.1.1.12  Transmit 
3.5. 1.1.2  Attributes 

3.5.1 .1.2.1  Oral 

3.5.1. 1.2.2  Written 


3.5. 1.2 


Meditation 
3.5. 1.2.1  Type 

3.5. 1.2.1. 1 


3.5. 1.2. 1.2 


3.5. 1.2. 1.3 


3. 5. 1.2. 2 Attributes 
3. 5.1. 2.2.1 
35.1.2.2.2 


Information  processing 
3. 5. 1.2. 1.1.1  Categorize 
3. 5. 1.2.  I I 2 Calculate 
3.51.2. 1.1.3  Code 
3. 5. 1.2. 1.1.4  Compute 
35.1.2.1.1.5  Interpolate 
3.5. 1 .2. 1.1.6  Itemize 
3.5.12. 1.1.7  Learn 

3.5. 1.2. 1.1. 8 Tabulate 

3. 5. 1.2. 1.1. 9 Translate 


Problem  solving  and  decision  making 

3.5. 1.2.1. 2.1  Analyse 

3.5. 1.2. 1.2.2  Deduce 

3.5. 1.2.1. 2.3  Induce 

3.5. 1.2. 1.2.4  Calculate 

3.5. 1.2. 1.2.5  Choose 

3.5. 1 .2. 1 .2.5. 1 Choose  from  known  alternatives 

3. 5. 1.2.1. 2.5.2  Choose  from  unknown  alternatives 

3.5. 1.2.1. 2.5.3  Choose  from  unspecified  alternatives 

3. 5. 1.2. 1.2.6  Compare 


3.5. 1.2. 1.2.6. 1 Order 


3. 5. 1.2. 1.2. 7 Compute 

3. 5. 1.2. 1.2.8  Estimate 

3. 5. 1.2. 1.2.9  Integrate 

3.5.1.2.1.2.10  Plan 

3.5.1.2.1.2.11  Supervise 
Recall 


3. 5. 1.2.1. 3. 1 Recall  facts 

3.5. 1.2. 1.3.2  Recall  principles 

3. 5. 1.2.1. 3. 3 Recall  procedures 

3. 5. 1.2. 1.3. 4 Timeshare 


Complexity 

Difficulty 


3 5.1.3  Motor  processes 

3.5. 1.3. 1 Type 

3.5. 1.3. 1.1  Complex-continuous 

35.1.3. 1.1.1  Adjust 
3.5.1.3.1.12  Align 

3. 5. 1.3. 1.1.3  Insert  object 

3. 5. 1.3. 1.1. 4 Regulate 

3.5. 1.3. 1.1.5  Remove  object 

3. 5. 1.3. 1.1. 6 Synchronize 
3.51.3. 1.1.7  Track 

3 5.1.3.1. 1.7.1  Visual  tracking  only 

3. 5.1.3. 1. 1.7.2  Visual  tracking  plus  position  plotting 

3. 5. 1.3. 1.1.8  Type  message  on  keyboard 

3. 5. 1.3. 1.1.9  Write 

3. 5. 1.3. 1.2  Compound 

3.5. 1.3. 1.3  Reflex 

3.5.1 .3. 1 .3. 1 Intersegmeotal 

3.5. 1.3. 1.3.2  Segmental 

3. 5. 1.3. 1.3.3  Suprasegmental 

3. 5. 1 .3. 1 .4  Simple-discrete 

3.5.1.3. 1.4.1  Activate 

3.5.1.3. 1.4.2  Close 

3 5.1.3.1.4.3  Connect 

3.5.1.3.1.4.4  Disconnect 
3.5.1.3.1.45  Join 

3.5. 1.3. 1.4.6  Move 

3.5.1.3.1.4.7  Press 

3.5. 1.31. 4 8 Set 

3. 5. 1.3. 1.4.9  Turn  single  rotary  control 

3.5. 1.3.2  Attributes 

3.5.1.3.2.1  Ballistic 

3.5.1. 3.2.2  Continuous 

3.5.1.3.2.3  Coordinated 

3.5. 1.3. 2. 4 Roe 
33.1.3.2.5  Gtm 
3.5. 1.3. 2. 6 Repetitive 
35  1527  MmHmi 

3.5.1.3.2.E  Static 

3. 5. 1.4  Perceptual  processing 

3.5. 1.4.1  Seardun^for  and  receiving  information 

3. 5.1.4. 1.1  Detea 

3. 5. 1.4. 1.1.1  Detea  non-verbal  cues 

3. 5. 1.4. 1.1. 2 Detea  verbal  cues 

3.5. 1.4. 1.2  Inspea 

3. 5. 1.4. 1.3  Observe 

3. 5. 1.4. 1.4  Read 

3.5. 1.4. 1.5  Receive 

3. 5. 1.4. 1.6  Scan 

3.5. 1.4. 1.7  Survey 

3.5. 1.4.2  Identifying  objects,  actions,  events 

3.5.1 .4.2. 1 Discriminate 

3. 5.1.4. 2.1.1  Discriminate  auditory  cues 

3.5. 1 .4.2. 1 .2  Discriminate  kinetic  cues 

3.5.1.4.2.1.3  Discriminate  non-verbal  cues 

3. 5. 1.4. 2. 1.4  Discriminate  tactile  cues 

3.5. 1.4. 2.1.5  Discriminate  verbal  cues 

3. 5.1.4. 2.1.6  Discriminate  visual  cues 

3.5.1.4.2.2  Identify 

3.5.1 .4.2.2. 1 Identify  non-verbal  cues 

3.5. 1.4. 2.2.2  Identify  verbal  cues 

3. 5.1.4. 2.3  Recognize 

3. 5.1. 4.2.3. 1 Recognize  non-verbal  cues 

3.5.1.4.2.3.2  Recognize  verbal  cues 

3.5.2  Attributes 

3.5.2. 1  Amount  of  labor  required 

35.2.2  Complexity 
3 5.2.3  Degree  of  response  chaining 

3.5.2.4  Difficulty 

3.5.2.5  Knowledge  of  results 

3.5.2.6  Output 

3.5.2.7  Pacing 

3 5.2.8  Precision 

3.5.2.9  Repetitiveness 

3.5.2.10  Skill  demands 

3.5.2. 1 1 Simultaneity  of  responses 

„ 3.5.2.12  Task  autonomy 


